[jira] [Commented] (HBASE-16264) Figure how to deal with endpoints and shaded pb
[ https://issues.apache.org/jira/browse/HBASE-16264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410500#comment-15410500 ] stack commented on HBASE-16264: --- Learnings: # We have a few Coprocessor Endpoints (CPEP) in the codebase currently scattered about. # Currently CPEP .protos that are in the codebase have their .protos mixed in with all the rest under hbase-protocol. But when we shade, all com.google.protobuf references are relocated. It means that there is a chasm between the client coming in on com.google.protobuf and our relocated org.apache.hadoop.hbase.shaded.com.google.protobuf.* references. # I tried to move all CPEP protos and supporting classes into a new hbase-endpoint module, one that does not have shading on it. The thought was that the requests in to Table on com.google.protobuf looking to run a CPEP c.g.p.*Service could travel down through the server as com.google.protobuf.*. Classes that dealt with CPEP had their imports adjusted to NOT be shaded references to protobuf. # I got pretty far but have now run into a wall. CPEP .protos, unfortunately, are not standalone; if they were, the above tactic would 'work'. What I mean by their not being standalone is that CPEPs will reference back into base HBase .protos so they can do Mutations and Results; e.g. MultiRowMutationProcessor imports Client.proto. But the contents of Client.proto will have been shaded and MultiRowMutationProcessor classes will not have been. The latter will be trying to pass to com.google.protobuf.* Types to the former Cilent.proto classes as params, etc., but since the Client.proto classes have been relocated, they are expecting o.h.h.shaded.c.g.p.* Types. I've run into the chasm again. Not sure how to bridge it. > Figure how to deal with endpoints and shaded pb > --- > > Key: HBASE-16264 > URL: https://issues.apache.org/jira/browse/HBASE-16264 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, Protobufs >Reporter: stack >Assignee: stack > > Come up w/ a migration plan for coprocessor endpoints when our pb is shaded. > Would be sweet if could make it so all just worked. At worst, come up w/ a > prescription for how to migrate existing CPs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-16321:
Resolution: Fixed
Hadoop Flags: Incompatible change
Release Note: HBase now ensures the jsr305 implementation from the findbugs
project is not included in its binary artifacts or the compile / runtime
dependencies of its user facing modules. Downstream users that rely on this jar
will need to update their dependencies.
Status: Resolved (was: Patch Available)
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-16263) Move all to do w/ protobuf -- *.proto files and generated classes -- under hbase-protocol
[ https://issues.apache.org/jira/browse/HBASE-16263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-16263. --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Committed. Resolving. > Move all to do w/ protobuf -- *.proto files and generated classes -- under > hbase-protocol > - > > Key: HBASE-16263 > URL: https://issues.apache.org/jira/browse/HBASE-16263 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-16263.master.001.patch > > > To better contain our protobuf exposure, keep all to do w/ protobuffing > inside the hbase-protocol module. > This task moves rest, examples, and server proto files that do testing and CP > endpoints etc., under hbase-protocol. The patch is big but it is all just > moving files. > interesting to see the copy-paste being done everywhere of proto generation. > For this reason alone, it is better to have all the protos in the one place > beside doc on how-to, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-9899: -- Attachment: HBASE-9899-addendum.patch Attach an addendum for master. When get nonce from mutation, it should decide if it has a nonce first. > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-9899-addendum.patch, HBASE-9899-branch-1.patch, > HBASE-9899-branch-1.patch, HBASE-9899-v1.patch, HBASE-9899-v2.patch, > HBASE-9899-v3.patch, HBASE-9899-v3.patch, HBASE-9899-v4.patch, > HBASE-9899-v4.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410469#comment-15410469 ] stack commented on HBASE-9899: -- Thanks > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-9899-branch-1.patch, HBASE-9899-branch-1.patch, > HBASE-9899-v1.patch, HBASE-9899-v2.patch, HBASE-9899-v3.patch, > HBASE-9899-v3.patch, HBASE-9899-v4.patch, HBASE-9899-v4.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-9899: -- Attachment: HBASE-9899-branch-1.patch Attath patch for branch-1 again to trigger HADOOP QA. > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-9899-branch-1.patch, HBASE-9899-branch-1.patch, > HBASE-9899-v1.patch, HBASE-9899-v2.patch, HBASE-9899-v3.patch, > HBASE-9899-v3.patch, HBASE-9899-v4.patch, HBASE-9899-v4.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16255) Backup/Restore IT
[ https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410458#comment-15410458 ] Ted Yu commented on HBASE-16255: [~dimaspivak]: Mind taking a look as well ? > Backup/Restore IT > - > > Key: HBASE-16255 > URL: https://issues.apache.org/jira/browse/HBASE-16255 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16255-v1.patch > > > Integration test for backup restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16255) Backup/Restore IT
[
https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410457#comment-15410457
]
Ted Yu commented on HBASE-16255:
{code}
+public class IntegrationTestBackupRestore {
{code}
Should the test extend IntegrationTestBase ?
{code}
+ protected static final TableName TABLE_NAME = TableName.valueOf(CLASS_NAME);
+ protected static final TableName TABLE_NAME_RESTORE =
TableName.valueOf(CLASS_NAME);
{code}
Should the table to restore to have different name ?
{code}
+ protected static final int DEFAULT_REGIONSERVER_COUNT = 2;
{code}
Please increase the number of region servers.
{code}
+// #6 - restore incremental backup for multiple tables, with overwrite
{code}
Did you intend to involve more than one table ?
Currently only one table is backed up.
> Backup/Restore IT
> -
>
> Key: HBASE-16255
> URL: https://issues.apache.org/jira/browse/HBASE-16255
> Project: HBase
> Issue Type: Task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Labels: backup
> Attachments: HBASE-16255-v1.patch
>
>
> Integration test for backup restore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16255) Backup/Restore IT
[ https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16255: --- Assignee: Vladimir Rodionov Labels: backup (was: ) > Backup/Restore IT > - > > Key: HBASE-16255 > URL: https://issues.apache.org/jira/browse/HBASE-16255 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16255-v1.patch > > > Integration test for backup restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410421#comment-15410421 ] Hudson commented on HBASE-16308: FAILURE: Integrated in HBase-Trunk_matrix #1362 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1362/]) Revert "HBASE-16308 Contain protobuf references Gather up the pb (stack: rev 0206dc67d643e4a248a319c724cd6e58f0e77603) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterKeepAliveConnection.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingTimeTracker.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientTimeouts.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mob/ExpiredMobFileCleaner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/TimeLimitedRpcController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/PayloadCarryingServerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java * hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientScanner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/Merge.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/Sweeper.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java * hbase-spark/src/main/java/org/apache/hadoop/hbase/spark/SparkSQLPushDownFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSimpleScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/MasterCoprocessorRpcChannel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/FlushRegionCallable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoadWithOldSecureEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/SecureBulkLoadClient.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ExpiredMobFileCleanerChore.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ExceptionUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicaWithCluster.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoadWithOldClient.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseIOException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RegionCoprocessorRpcChannel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestNamespace.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 >
[jira] [Commented] (HBASE-15756) Pluggable RpcServer
[ https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410412#comment-15410412 ] stack commented on HBASE-15756: --- Testing today, the results are less conclusive. I can do about 380k/sec w/ current rpcserver and then just over with 12 workers on the netty server (415k), so a difference of almost 10%. I can't get to 450 today for some reason... not sure what I was doing yesterday evening. > Pluggable RpcServer > --- > > Key: HBASE-15756 > URL: https://issues.apache.org/jira/browse/HBASE-15756 > Project: HBase > Issue Type: Improvement > Components: Performance, rpc >Reporter: binlijin >Assignee: binlijin >Priority: Critical > Attachments: Netty4RpcServer_forperf.patch, NettyRpcServer.patch, > NettyRpcServer_forperf.patch, PooledByteBufAllocator.patch, > PooledByteBufAllocator2.patch, gc.png, gets.png, gets.png, idle.png, > patched.vs.patched_and_cached.vs.no_patch.png, queue.png > > > Current we use a simple RpcServer, and can not configure and use other > implementation.This issue is to make the RpcServer pluggable, so we can make > other implementation for example netty rpc server. Patch will upload laterly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410409#comment-15410409
]
Hudson commented on HBASE-16321:
SUCCESS: Integrated in HBase-1.2 #695 (See
[https://builds.apache.org/job/HBase-1.2/695/])
HBASE-16321 ensure no findbugs-jsr305 (busbey: rev
7b91c136b238960ba2ba9ab985b62db1ae5f674c)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
* hbase-client/pom.xml
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormat.java
* pom.xml
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
* hbase-server/pom.xml
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16340) ensure no Xerces jars included
[ https://issues.apache.org/jira/browse/HBASE-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410390#comment-15410390 ] Hudson commented on HBASE-16340: FAILURE: Integrated in HBase-1.3 #810 (See [https://builds.apache.org/job/HBase-1.3/810/]) HBASE-16340 exclude Xerces iplementation jars from coming in (busbey: rev 262a4e666fce5eceb61f077c13b0e623f532598a) * pom.xml > ensure no Xerces jars included > -- > > Key: HBASE-16340 > URL: https://issues.apache.org/jira/browse/HBASE-16340 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16340.1.patch, HBASE-16340.2.patch > > > When we moved our pom to Hadoop 2.7 we picked up a transitive Xerces > implementation. We should exclude this to ensure we don't get a conflict with > the implementation that ships with the jvm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8386) deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
[
https://issues.apache.org/jira/browse/HBASE-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410392#comment-15410392
]
Hudson commented on HBASE-8386:
---
FAILURE: Integrated in HBase-1.3 #810 (See
[https://builds.apache.org/job/HBase-1.3/810/])
HBASE-8386 deprecate TableMapReduce.addDependencyJars(Configuration, (busbey:
rev 3175d8073071854b3940c4dc547fbe9edaa234bf)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java
* src/main/asciidoc/_chapters/mapreduce.adoc
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestWithCellVisibilityLoadAndVerify.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestLoadAndVerify.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestTableSnapshotInputFormat.java
> deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
> ---
>
> Key: HBASE-8386
> URL: https://issues.apache.org/jira/browse/HBASE-8386
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-8386.1.patch
>
>
> We expose two public static methods names {{addDependencyJars}}. One of them,
> {{void addDependencyJars(Job}}, is very helpful -- goes out of its way to
> detect job dependencies as well as shipping all the necessary HBase
> dependencies. The other is shfty and nefarious, {{void
> addDependencyJars(Configuration, Class...)}} -- it only adds exactly what
> the user requests, forcing them to resolve dependencies themselves and giving
> a false sense of security. We should deprecate the latter throw a big giant
> warning when people use that one. The handy functionality of providing help
> when our heuristics fail can be added via a new method signature, something
> like {{void addDependencyJars(Job, Class ...}}. This method would do
> everything {{void addDependencyJars(Job}} does, plus let the user specify
> arbitrary additional classes. That way HBase still can help the user, but
> also gives them super-powers to compensate for when our heuristics fail.
> For reference, this appears to be the reason why HBase + Pig doesn't really
> work out of the box. See
> [HBaseStorage.java|https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java#L730]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410389#comment-15410389
]
Hudson commented on HBASE-16321:
FAILURE: Integrated in HBase-1.3 #810 (See
[https://builds.apache.org/job/HBase-1.3/810/])
HBASE-16321 ensure no findbugs-jsr305 (busbey: rev
bf12fe6788b7f6e2d8568177ffaf904662ac6d0e)
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
* pom.xml
* hbase-server/pom.xml
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormat.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
* hbase-client/pom.xml
*
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16303) FilterList with MUST_PASS_ONE optimization
[ https://issues.apache.org/jira/browse/HBASE-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410391#comment-15410391 ] Hudson commented on HBASE-16303: FAILURE: Integrated in HBase-1.3 #810 (See [https://builds.apache.org/job/HBase-1.3/810/]) Revert "HBASE-16303 Addendum as the wrong patch was committed" (busbey: rev 76658440dfaddde890a216c8247855182a6c9ac5) * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java > FilterList with MUST_PASS_ONE optimization > -- > > Key: HBASE-16303 > URL: https://issues.apache.org/jira/browse/HBASE-16303 > Project: HBase > Issue Type: Bug > Components: Filters >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16303.patch, HBASE-16303.patch, > HBASE-16303_1.patch, HBASE-16303_1.patch > > > This JIRA was created to add @Test tag for all the filter related tests. > After doing that one such test failed in > TestFilterList#testFilterListTwoFiltersMustPassOne. > The test assumes that when a PrefixFilter is added to a filterlist once the > given prefix is passed we should SKIP all other rows. But the impl in > FilterList for MUST_PASS_ONE filter is such that once the filter in the list > sees that filterAllRemaining is true we don't use that to decide if to to > SEE_NEXT_USING_HINT or not. This patch tries to handle that case where if for > a filter FilterAllRemaining is true then SEEK_NEXT_USING_HINT need not be > done. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15866: --- Fix Version/s: 1.4.0 I've done some poking around with 0.98. The code is significantly different. RegionServerCallable and RpcRetryingCaller implementations differ, especially RegionServerCallable#prepare (in 0.98 we use separate callers with rpc settings derived from connection not table to do region relocation, so we don't correctly honor the table level settings) and RpcRetryingCaller#callWithRetry (uses a thread local for setting rpc timeouts in 0.98, later versions don't). Setting a RPC timeout in configuration works. Both connection and table pick it up. Using only the setter on HTable doesn't. Only the callables created by the table use the setting; others created by the HConnection do not. I'm not saying it can't be done, but it's much easier to achieve this with master and branch-1, and why not have another reason to move off of 0.98. Updated fix versions accordingly, only targeting master and branch-1. If someone feels differently about porting this back to 0.98, by all means open another issue for backport and tackle it there. > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15866.patch, HBASE-15866.patch, HBASE-15866.patch, > read-write-rpc-timeouts.patch, read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410373#comment-15410373 ] Andrew Purtell commented on HBASE-15866: Thanks [~vivek.kopp...@gmail.com], let me try it out > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15866.patch, HBASE-15866.patch, HBASE-15866.patch, > read-write-rpc-timeouts.patch, read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16366) Restore operation into new table can fail
Vladimir Rodionov created HBASE-16366: - Summary: Restore operation into new table can fail Key: HBASE-16366 URL: https://issues.apache.org/jira/browse/HBASE-16366 Project: HBase Issue Type: Bug Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov If restore from backup into new table we need to make sure that new table is available online. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16255) Backup/Restore IT
[ https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-16255: -- Attachment: HBASE-16255-v1.patch Patch v1. cc: [~tedyu] > Backup/Restore IT > - > > Key: HBASE-16255 > URL: https://issues.apache.org/jira/browse/HBASE-16255 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov > Attachments: HBASE-16255-v1.patch > > > Integration test for backup restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15945) Patch for Cell
[ https://issues.apache.org/jira/browse/HBASE-15945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410365#comment-15410365 ] Enis Soztutar commented on HBASE-15945: --- Thanks, should have git pull'ed first. > Patch for Cell > -- > > Key: HBASE-15945 > URL: https://issues.apache.org/jira/browse/HBASE-15945 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-15945-HBASE-14850.v2.patch, > HBASE-15945-HBASE-14850.v3.patch, HBASE-15945-HBASE-14850.v4.patch, > HBASE-15945.HBASE-14850.v1.patch, HBASE-15945.HBASE-14850.v5.patch, > HBASE-15945.HBASE-14850.v6.patch > > > This patch contains an implementation of Key Value, Bytes and Cell modeled on > the lines of Java implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410361#comment-15410361 ] Enis Soztutar commented on HBASE-16148: --- bq. Should we update the server clock with client set timestamps ? In HBase, client can set any timestamp he wants, it could have any skew. I feel we should not update the clock with client set timestamps, rather send another field in put request which just reads time of system clock, use that to update server clock. With HLC the expectation is that the client normally will not set the timestamps, only servers will do. But from RS perspective there is no distinction between doing Puts where the client has actually set the timestamps, versus through intercluster replication or wal recovery receiving Puts with already-set timestamps. It would be good to differentiate between these two cases so that if it is the client, we will not call update server clock, otherwise we will. However, if we want to be flexible still in having the client to optionally set timestamps together with servers, then we can do it so that all Puts update the server clocks. If the request is more than the configured clock skew, it will be rejected anyway. > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.3.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15945) Patch for Cell
[ https://issues.apache.org/jira/browse/HBASE-15945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410360#comment-15410360 ] Elliott Clark commented on HBASE-15945: --- https://github.com/apache/hbase/commit/fa3ab420d19bba853d96576176257b5c12f1a48f Isn't that it ? It appears to be there on my git checkout ? > Patch for Cell > -- > > Key: HBASE-15945 > URL: https://issues.apache.org/jira/browse/HBASE-15945 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-15945-HBASE-14850.v2.patch, > HBASE-15945-HBASE-14850.v3.patch, HBASE-15945-HBASE-14850.v4.patch, > HBASE-15945.HBASE-14850.v1.patch, HBASE-15945.HBASE-14850.v5.patch, > HBASE-15945.HBASE-14850.v6.patch > > > This patch contains an implementation of Key Value, Bytes and Cell modeled on > the lines of Java implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15945) Patch for Cell
[ https://issues.apache.org/jira/browse/HBASE-15945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410358#comment-15410358 ] Enis Soztutar commented on HBASE-15945: --- I thought [~eclark] pushed it, but it does not seem to be there in branch HBASE-14850. > Patch for Cell > -- > > Key: HBASE-15945 > URL: https://issues.apache.org/jira/browse/HBASE-15945 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-15945-HBASE-14850.v2.patch, > HBASE-15945-HBASE-14850.v3.patch, HBASE-15945-HBASE-14850.v4.patch, > HBASE-15945.HBASE-14850.v1.patch, HBASE-15945.HBASE-14850.v5.patch, > HBASE-15945.HBASE-14850.v6.patch > > > This patch contains an implementation of Key Value, Bytes and Cell modeled on > the lines of Java implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15756) Pluggable RpcServer
[ https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410355#comment-15410355 ] binlijin commented on HBASE-15756: -- So the Netty's NioWork doing more work, and the number may be should be more then current rpcserver's reader number. > Pluggable RpcServer > --- > > Key: HBASE-15756 > URL: https://issues.apache.org/jira/browse/HBASE-15756 > Project: HBase > Issue Type: Improvement > Components: Performance, rpc >Reporter: binlijin >Assignee: binlijin >Priority: Critical > Attachments: Netty4RpcServer_forperf.patch, NettyRpcServer.patch, > NettyRpcServer_forperf.patch, PooledByteBufAllocator.patch, > PooledByteBufAllocator2.patch, gc.png, gets.png, gets.png, idle.png, > patched.vs.patched_and_cached.vs.no_patch.png, queue.png > > > Current we use a simple RpcServer, and can not configure and use other > implementation.This issue is to make the RpcServer pluggable, so we can make > other implementation for example netty rpc server. Patch will upload laterly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15756) Pluggable RpcServer
[ https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410352#comment-15410352 ] binlijin commented on HBASE-15756: -- [~stack] i am not use the Netty4RpcServer much, so may be i encounter, but also not dig in. > Pluggable RpcServer > --- > > Key: HBASE-15756 > URL: https://issues.apache.org/jira/browse/HBASE-15756 > Project: HBase > Issue Type: Improvement > Components: Performance, rpc >Reporter: binlijin >Assignee: binlijin >Priority: Critical > Attachments: Netty4RpcServer_forperf.patch, NettyRpcServer.patch, > NettyRpcServer_forperf.patch, PooledByteBufAllocator.patch, > PooledByteBufAllocator2.patch, gc.png, gets.png, gets.png, idle.png, > patched.vs.patched_and_cached.vs.no_patch.png, queue.png > > > Current we use a simple RpcServer, and can not configure and use other > implementation.This issue is to make the RpcServer pluggable, so we can make > other implementation for example netty rpc server. Patch will upload laterly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15756) Pluggable RpcServer
[
https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410348#comment-15410348
]
binlijin commented on HBASE-15756:
--
{quote}
"But our current rpcserver, the reader only read request from client, write
response to client is done by handler or Responder."
In my observation, the handler seems to be able to put the response on the wire
itself. Rare do we go via the queue and then the single Responder thread. So I
am not sure why netty is doing better (but am happy it does).
{quote}
[~stack] Yes, i am just point out the different.
> Pluggable RpcServer
> ---
>
> Key: HBASE-15756
> URL: https://issues.apache.org/jira/browse/HBASE-15756
> Project: HBase
> Issue Type: Improvement
> Components: Performance, rpc
>Reporter: binlijin
>Assignee: binlijin
>Priority: Critical
> Attachments: Netty4RpcServer_forperf.patch, NettyRpcServer.patch,
> NettyRpcServer_forperf.patch, PooledByteBufAllocator.patch,
> PooledByteBufAllocator2.patch, gc.png, gets.png, gets.png, idle.png,
> patched.vs.patched_and_cached.vs.no_patch.png, queue.png
>
>
> Current we use a simple RpcServer, and can not configure and use other
> implementation.This issue is to make the RpcServer pluggable, so we can make
> other implementation for example netty rpc server. Patch will upload laterly
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16365) Hook up Put work flow
Elliott Clark created HBASE-16365: - Summary: Hook up Put work flow Key: HBASE-16365 URL: https://issues.apache.org/jira/browse/HBASE-16365 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Koppuru updated HBASE-15866: -- Attachment: HBASE-15866.patch Contains changes on the latest master branch > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, HBASE-15866.patch, HBASE-15866.patch, > read-write-rpc-timeouts.patch, read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16198) Enhance backup history command
[ https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16198: --- Labels: backup (was: ) > Enhance backup history command > -- > > Key: HBASE-16198 > URL: https://issues.apache.org/jira/browse/HBASE-16198 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch, > HBASE-16198-v3.patch, HBASE-16198-v4.patch, HBASE-16198-v5.patch > > > We need history per table and ability to run command on a fully disabled > cluster (using info from backup file system only). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16198) Enhance backup history command
[ https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-16198. Resolution: Fixed Hadoop Flags: Reviewed > Enhance backup history command > -- > > Key: HBASE-16198 > URL: https://issues.apache.org/jira/browse/HBASE-16198 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch, > HBASE-16198-v3.patch, HBASE-16198-v4.patch, HBASE-16198-v5.patch > > > We need history per table and ability to run command on a fully disabled > cluster (using info from backup file system only). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[
https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410290#comment-15410290
]
Hadoop QA commented on HBASE-15866:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 59s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 4s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
11s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 2m 14s
{color} | {color:red} hbase-client-jdk1.8.0 with JDK v1.8.0 generated 1 new +
13 unchanged - 0 fixed = 14 total (was 13) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 3m 38s
{color} | {color:red} hbase-client-jdk1.7.0_80 with JDK v1.7.0_80 generated 1
new + 13 unchanged - 0 fixed = 14 total (was 13) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 46s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 36s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 14s {color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410289#comment-15410289
]
Hudson commented on HBASE-16321:
SUCCESS: Integrated in HBase-1.2-IT #573 (See
[https://builds.apache.org/job/HBase-1.2-IT/573/])
HBASE-16321 ensure no findbugs-jsr305 (busbey: rev
7b91c136b238960ba2ba9ab985b62db1ae5f674c)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormat.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
* hbase-server/pom.xml
*
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
* pom.xml
* hbase-client/pom.xml
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16198) Enhance backup history command
[ https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410281#comment-15410281 ] Ted Yu commented on HBASE-16198: Looks like TestBackupSystemTable#testBackupHistory() should be covered by other two tests. > Enhance backup history command > -- > > Key: HBASE-16198 > URL: https://issues.apache.org/jira/browse/HBASE-16198 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch, > HBASE-16198-v3.patch, HBASE-16198-v4.patch, HBASE-16198-v5.patch > > > We need history per table and ability to run command on a fully disabled > cluster (using info from backup file system only). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16340) ensure no Xerces jars included
[ https://issues.apache.org/jira/browse/HBASE-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410249#comment-15410249 ] Hudson commented on HBASE-16340: SUCCESS: Integrated in HBase-1.3-IT #782 (See [https://builds.apache.org/job/HBase-1.3-IT/782/]) HBASE-16340 exclude Xerces iplementation jars from coming in (busbey: rev 262a4e666fce5eceb61f077c13b0e623f532598a) * pom.xml > ensure no Xerces jars included > -- > > Key: HBASE-16340 > URL: https://issues.apache.org/jira/browse/HBASE-16340 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16340.1.patch, HBASE-16340.2.patch > > > When we moved our pom to Hadoop 2.7 we picked up a transitive Xerces > implementation. We should exclude this to ensure we don't get a conflict with > the implementation that ships with the jvm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16303) FilterList with MUST_PASS_ONE optimization
[ https://issues.apache.org/jira/browse/HBASE-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410250#comment-15410250 ] Hudson commented on HBASE-16303: SUCCESS: Integrated in HBase-1.3-IT #782 (See [https://builds.apache.org/job/HBase-1.3-IT/782/]) Revert "HBASE-16303 Addendum as the wrong patch was committed" (busbey: rev 76658440dfaddde890a216c8247855182a6c9ac5) * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java > FilterList with MUST_PASS_ONE optimization > -- > > Key: HBASE-16303 > URL: https://issues.apache.org/jira/browse/HBASE-16303 > Project: HBase > Issue Type: Bug > Components: Filters >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16303.patch, HBASE-16303.patch, > HBASE-16303_1.patch, HBASE-16303_1.patch > > > This JIRA was created to add @Test tag for all the filter related tests. > After doing that one such test failed in > TestFilterList#testFilterListTwoFiltersMustPassOne. > The test assumes that when a PrefixFilter is added to a filterlist once the > given prefix is passed we should SKIP all other rows. But the impl in > FilterList for MUST_PASS_ONE filter is such that once the filter in the list > sees that filterAllRemaining is true we don't use that to decide if to to > SEE_NEXT_USING_HINT or not. This patch tries to handle that case where if for > a filter FilterAllRemaining is true then SEEK_NEXT_USING_HINT need not be > done. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8386) deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
[
https://issues.apache.org/jira/browse/HBASE-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410251#comment-15410251
]
Hudson commented on HBASE-8386:
---
SUCCESS: Integrated in HBase-1.3-IT #782 (See
[https://builds.apache.org/job/HBase-1.3-IT/782/])
HBASE-8386 deprecate TableMapReduce.addDependencyJars(Configuration, (busbey:
rev 3175d8073071854b3940c4dc547fbe9edaa234bf)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestLoadAndVerify.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestTableSnapshotInputFormat.java
* src/main/asciidoc/_chapters/mapreduce.adoc
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestWithCellVisibilityLoadAndVerify.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
> deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
> ---
>
> Key: HBASE-8386
> URL: https://issues.apache.org/jira/browse/HBASE-8386
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-8386.1.patch
>
>
> We expose two public static methods names {{addDependencyJars}}. One of them,
> {{void addDependencyJars(Job}}, is very helpful -- goes out of its way to
> detect job dependencies as well as shipping all the necessary HBase
> dependencies. The other is shfty and nefarious, {{void
> addDependencyJars(Configuration, Class...)}} -- it only adds exactly what
> the user requests, forcing them to resolve dependencies themselves and giving
> a false sense of security. We should deprecate the latter throw a big giant
> warning when people use that one. The handy functionality of providing help
> when our heuristics fail can be added via a new method signature, something
> like {{void addDependencyJars(Job, Class ...}}. This method would do
> everything {{void addDependencyJars(Job}} does, plus let the user specify
> arbitrary additional classes. That way HBase still can help the user, but
> also gives them super-powers to compensate for when our heuristics fail.
> For reference, this appears to be the reason why HBase + Pig doesn't really
> work out of the box. See
> [HBaseStorage.java|https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java#L730]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410248#comment-15410248
]
Hudson commented on HBASE-16321:
SUCCESS: Integrated in HBase-1.3-IT #782 (See
[https://builds.apache.org/job/HBase-1.3-IT/782/])
HBASE-16321 ensure no findbugs-jsr305 (busbey: rev
bf12fe6788b7f6e2d8568177ffaf904662ac6d0e)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
* pom.xml
* hbase-server/pom.xml
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormat.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
* hbase-client/pom.xml
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Status: Patch Available (was: Open) > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.3.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Status: Open (was: Patch Available) > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.3.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16198) Enhance backup history command
[
https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410240#comment-15410240
]
Ted Yu commented on HBASE-16198:
+ public void testBackupHistory() throws Exception {
There are 3 sub-tests with the same name:
TestBackupShowHistoryFromBackupDestination, TestBackupShowHistory and
TestBackupSystemTable
Is it possible to consolidate them ?
> Enhance backup history command
> --
>
> Key: HBASE-16198
> URL: https://issues.apache.org/jira/browse/HBASE-16198
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch,
> HBASE-16198-v3.patch, HBASE-16198-v4.patch, HBASE-16198-v5.patch
>
>
> We need history per table and ability to run command on a fully disabled
> cluster (using info from backup file system only).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Attachment: HLC.3.patch > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.3.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15866: --- Status: Open (was: Patch Available) Cancelling patch. Latest patch needs a rebase on latest master. Set Patch Available once rebase is done > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, HBASE-15866.patch, > read-write-rpc-timeouts.patch, read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16198) Enhance backup history command
[ https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-16198: -- Attachment: HBASE-16198-v5.patch v5. fixed typo in the code. > Enhance backup history command > -- > > Key: HBASE-16198 > URL: https://issues.apache.org/jira/browse/HBASE-16198 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch, > HBASE-16198-v3.patch, HBASE-16198-v4.patch, HBASE-16198-v5.patch > > > We need history per table and ability to run command on a fully disabled > cluster (using info from backup file system only). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16198) Enhance backup history command
[ https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410201#comment-15410201 ] Vladimir Rodionov commented on HBASE-16198: --- Yes, you are right. > Enhance backup history command > -- > > Key: HBASE-16198 > URL: https://issues.apache.org/jira/browse/HBASE-16198 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch, > HBASE-16198-v3.patch, HBASE-16198-v4.patch > > > We need history per table and ability to run command on a fully disabled > cluster (using info from backup file system only). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16198) Enhance backup history command
[
https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410187#comment-15410187
]
Ted Yu commented on HBASE-16198:
{code}
+private Path getBackupRootPath() {
+ String value = cmdline.getOptionValue("t");
+ if (value == null) return null;
+ return new Path(value);
+}
+
+private TableName getTableName() {
+ String value = cmdline.getOptionValue("t");
{code}
I don't think backup root should be using the same option value as that of
table name.
> Enhance backup history command
> --
>
> Key: HBASE-16198
> URL: https://issues.apache.org/jira/browse/HBASE-16198
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch,
> HBASE-16198-v3.patch, HBASE-16198-v4.patch
>
>
> We need history per table and ability to run command on a fully disabled
> cluster (using info from backup file system only).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410176#comment-15410176 ] stack commented on HBASE-16308: --- Sorry about that [~apurtell]. Let me revert till I figure the failing test. Looked good in the test runs above. Must have skipped a bunch. Will look. > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-16308.master.001.patch, > HBASE-16308.master.002.patch, HBASE-16308.master.003.patch, > HBASE-16308.master.004.patch, HBASE-16308.master.005.patch, > HBASE-16308.master.006.patch, HBASE-16308.master.006.patch > > > Clean up our protobuf references so contained to just a few classes rather > than being spread about the codebase. Doing this work will make it easier > landing the parent issue and will make it more clear where the division > between shaded protobuf and unshaded protobuf lies (we need to continue with > unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint > Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-16321:
Fix Version/s: 0.98.21
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-16308) Contain protobuf references
[
https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-16308:
This change breaks - at least - TestIncrementsFromClientSide and
TestIncrementFromClientSideWithCoprocessor
{noformat}
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 25.569 sec <<<
FAILURE! - in
org.apache.hadoop.hbase.client.TestIncrementFromClientSideWithCoprocessor
testDuplicateIncrement(org.apache.hadoop.hbase.client.TestIncrementFromClientSideWithCoprocessor)
Time elapsed: 3.936 sec <<< FAILURE!
java.lang.AssertionError: Expected value [1] Got value [2]
{noformat}
> Contain protobuf references
> ---
>
> Key: HBASE-16308
> URL: https://issues.apache.org/jira/browse/HBASE-16308
> Project: HBase
> Issue Type: Sub-task
> Components: Protobufs
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: HBASE-16308.master.001.patch,
> HBASE-16308.master.002.patch, HBASE-16308.master.003.patch,
> HBASE-16308.master.004.patch, HBASE-16308.master.005.patch,
> HBASE-16308.master.006.patch, HBASE-16308.master.006.patch
>
>
> Clean up our protobuf references so contained to just a few classes rather
> than being spread about the codebase. Doing this work will make it easier
> landing the parent issue and will make it more clear where the division
> between shaded protobuf and unshaded protobuf lies (we need to continue with
> unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint
> Coprocessors)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410169#comment-15410169 ] Andrew Purtell commented on HBASE-16308: If I go one rev back from this change, it's good. At this change, it's bad > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-16308.master.001.patch, > HBASE-16308.master.002.patch, HBASE-16308.master.003.patch, > HBASE-16308.master.004.patch, HBASE-16308.master.005.patch, > HBASE-16308.master.006.patch, HBASE-16308.master.006.patch > > > Clean up our protobuf references so contained to just a few classes rather > than being spread about the codebase. Doing this work will make it easier > landing the parent issue and will make it more clear where the division > between shaded protobuf and unshaded protobuf lies (we need to continue with > unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint > Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410162#comment-15410162
]
Appy commented on HBASE-14345:
--
[~reidchan], the patch you have uploaded is raw patch and doesn't contain
information like author, patch description,etc.
Please use dev-support/submit-patch.py or format-patch.sh.
submit-patch.py will create and upload the patch to jira. Make sure to read
help. It takes 10-15 min to setup but will makes things very easy for future.
> Consolidate printUsage in IntegrationTestLoadAndVerify
> --
>
> Key: HBASE-14345
> URL: https://issues.apache.org/jira/browse/HBASE-14345
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Reid Chan
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-14345.002.patch, HBASE-14345.003.patch,
> HBASE-14345.004.patch, HBASE-14345.patch, itlav-2016-07-07.png, itlv.png
>
>
> Investigating the use of {{itlav}} is a little screwy. Subclasses are not
> overriding the {{printUsage()}} methods correctly, so you have to pass
> {{--help}} to get some info and no arguments to get the rest.
> {noformat}
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify --help
> usage: bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
>
> Options:
> -h,--help Show usage
> -m,--monkey Which chaos monkey to run
> -monkeyProps The properties file for specifying chaos monkey
> properties.
> -ncc,--noClusterCleanUp Don't clean up the cluster at the end
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
> IntegrationTestLoadAndVerify [-Doptions]
> Loads a table with row dependencies and verifies the dependency chains
> Options
> -Dloadmapper.table=Table to write/verify (default autogen)
> -Dloadmapper.backrefs=Number of backreferences per row (default
> 50)
> -Dloadmapper.num_to_write=Number of rows per mapper (default 100,000
> per mapper)
> -Dloadmapper.deleteAfter= Delete after a successful verify (default
> true)
> -Dloadmapper.numPresplits=Number of presplit regions to start with
> (default 40)
> -Dloadmapper.map.tasks= Number of map tasks for load (default 200)
> -Dverify.reduce.tasks=Number of reduce tasks for verify (default
> 35)
> -Dverify.scannercaching= Number hbase scanner caching rows to read
> (default 50)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[
https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-15866:
---
Attachment: HBASE-15866.patch
Patch for master rebased on recent changes.
Reviewboard: https://reviews.apache.org/r/50869/
Also fixes TestHCM#testCallableSleep
The following methods are *added* to Table:
{code}
int getReadRpcTimeout();
void setReadRpcTimeout(int readRpcTimeout);
int getWriteRpcTimeout();
void setWriteRpcTimeout(int writeRpcTimeout);
{code}
The following methods are *deprecated* in Table:
{code}
int getRpcTimeout();
void setRpcTimeout(int rpcTimeout);
{code}
This change targets master, branch-1, and 0.98.
In 0.98, we will not be changing HTableInterface but will be adding the new
methods to HTable.
> Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
> ---
>
> Key: HBASE-15866
> URL: https://issues.apache.org/jira/browse/HBASE-15866
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Andrew Purtell
>Assignee: Vivek Koppuru
> Labels: newbie, patch
> Fix For: 2.0.0
>
> Attachments: HBASE-15866.patch, HBASE-15866.patch,
> read-write-rpc-timeouts.patch, read-write-rpc.v1.patch
>
>
> We have a single tunable for the RPC timeout interval - hbase.rpc.timeout.
> This is fine for the general case but there are use cases where it would be
> advantageous to set two separate timeouts for reads (gets, scans, perhaps
> with significant server side filtering - although the new scanner heartbeat
> feature mitigates where available) and mutations (fail fast under tight SLA,
> resubmit or take mitigating action).
> I propose we refer to a configuration setting "hbase.rpc.read.timeout" when
> handling read operations and "hbase.rpc.write.timeout" when handling write
> operations. If those values are not set in the configuration, fall back to
> the value of "hbase.rpc.timeout" or its default.
> So for example in HTable instead of one global timeout for each RPC
> (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up
> in HTable#finishSetup. Then wherever we set up RPC with
> RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or
> write timeout depending on what the op is.
> In general I don't like the idea of adding configuration parameters to our
> already heavyweight set, but I think the inability to control timeouts
> separately for reads and writes is an operational deficit.
> See also PHOENIX-2916.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410137#comment-15410137
]
Hudson commented on HBASE-16321:
FAILURE: Integrated in HBase-1.4 #335 (See
[https://builds.apache.org/job/HBase-1.4/335/])
HBASE-16321 ensure no findbugs-jsr305 (busbey: rev
a5f98c3f9ede67f3a5fb76bd00450f17ca233207)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormat.java
* pom.xml
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
* hbase-server/pom.xml
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
* hbase-client/pom.xml
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16359) NullPointerException in RSRpcServices.openRegion()
[
https://issues.apache.org/jira/browse/HBASE-16359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410138#comment-15410138
]
Hudson commented on HBASE-16359:
FAILURE: Integrated in HBase-1.4 #335 (See
[https://builds.apache.org/job/HBase-1.4/335/])
HBASE-16359 Addendum fixes TestZKBasedOpenCloseRegion (tedyu: rev
b1c60940da7035ce1473a11fb5aa0abdccb8d382)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
> NullPointerException in RSRpcServices.openRegion()
> --
>
> Key: HBASE-16359
> URL: https://issues.apache.org/jira/browse/HBASE-16359
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16359.addendum, 16359.v2.txt
>
>
> I was investigating why some region failed to move out of transition within
> timeout 12ms and found the following in region server log:
> {code}
> 2016-08-04 09:19:52,616 INFO
> [B.priority.fifo.QRpcServer.handler=12,queue=0,port=16020]
> regionserver.RSRpcServices: Open hbck_table_772674,,1470302211047.
> da859880bb51bc0fd25979798a96c444.
> 2016-08-04 09:19:52,620 ERROR
> [B.priority.fifo.QRpcServer.handler=12,queue=0,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:1530)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22737)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2127)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> {code}
> Here is related code - NPE was thrown from the last line:
> {code}
> htd = htds.get(region.getTable());
> if (htd == null) {
> htd = regionServer.tableDescriptors.get(region.getTable());
> htds.put(region.getTable(), htd);
> }
> ...
> if (region.isMetaRegion()) {
> regionServer.service.submit(new OpenMetaHandler(
> regionServer, regionServer, region, htd, masterSystemTime,
> coordination, ord));
> } else {
>
> regionServer.updateRegionFavoredNodesMapping(region.getEncodedName(),
> regionOpenInfo.getFavoredNodesList());
> if (htd.getPriority() >= HConstants.ADMIN_QOS ||
> region.getTable().isSystemTable()) {
> {code}
> region.getTable() shouldn't be null since it is called via
> htds.get(region.getTable()) unconditionally.
> It seems htd was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410086#comment-15410086 ] Andrew Purtell commented on HBASE-15866: [~vivek.kopp...@gmail.com] your latest patch fails to apply to master, with 6 rejects in HTable.java. > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, read-write-rpc-timeouts.patch, > read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410081#comment-15410081 ] Andrew Purtell edited comment on HBASE-15866 at 8/5/16 9:18 PM: >From [~vivek.kopp...@gmail.com]: Fixed broken unit test and added unit tests for TestHCM#testWriteRpcTimeout and TestHCM#testReadRpcTimeout, along with necessary changes to make that work. This is the patch labelled "read-write-rpc-timeouts-edited.v2.patch" was (Author: apurtell): Fixed broken unit test and added unit tests for TestHCM#testWriteRpcTimeout and TestHCM#testReadRpcTimeout, along with necessary changes to make that work. This is the patch labelled "read-write-rpc-timeouts-edited.v2.patch" > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, read-write-rpc-timeouts.patch, > read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[
https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410067#comment-15410067
]
Hadoop QA commented on HBASE-15866:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15866 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822373/read-write-rpc.v1.patch
|
| JIRA Issue | HBASE-15866 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2968/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
> ---
>
> Key: HBASE-15866
> URL: https://issues.apache.org/jira/browse/HBASE-15866
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Andrew Purtell
>Assignee: Vivek Koppuru
> Labels: newbie, patch
> Fix For: 2.0.0
>
> Attachments: HBASE-15866.patch, read-write-rpc-timeouts.patch,
> read-write-rpc.v1.patch
>
>
> We have a single tunable for the RPC timeout interval - hbase.rpc.timeout.
> This is fine for the general case but there are use cases where it would be
> advantageous to set two separate timeouts for reads (gets, scans, perhaps
> with significant server side filtering - although the new scanner heartbeat
> feature mitigates where available) and mutations (fail fast under tight SLA,
> resubmit or take mitigating action).
> I propose we refer to a configuration setting "hbase.rpc.read.timeout" when
> handling read operations and "hbase.rpc.write.timeout" when handling write
> operations. If those values are not set in the configuration, fall back to
> the value of "hbase.rpc.timeout" or its default.
> So for example in HTable instead of one global timeout for each RPC
> (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up
> in HTable#finishSetup. Then wherever we set up RPC with
> RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or
> write timeout depending on what the op is.
> In general I don't like the idea of adding configuration parameters to our
> already heavyweight set, but I think the inability to control timeouts
> separately for reads and writes is an operational deficit.
> See also PHOENIX-2916.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Koppuru updated HBASE-15866: -- Attachment: (was: read-write-rpc-timeouts-edited.v2.patch) > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, read-write-rpc-timeouts.patch, > read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Koppuru updated HBASE-15866: -- Attachment: read-write-rpc.v1.patch > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, read-write-rpc-timeouts.patch, > read-write-rpc.v1.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-8386) deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
[
https://issues.apache.org/jira/browse/HBASE-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-8386.
Resolution: Fixed
Release Note: The MapReduce helper function
`TableMapReduce.addDependencyJars(Configuration, class ...)` has been
deprecated since it is easy to use incorrectly. Most users should rely on
addDependencyJars(Job) instead.
> deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
> ---
>
> Key: HBASE-8386
> URL: https://issues.apache.org/jira/browse/HBASE-8386
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-8386.1.patch
>
>
> We expose two public static methods names {{addDependencyJars}}. One of them,
> {{void addDependencyJars(Job}}, is very helpful -- goes out of its way to
> detect job dependencies as well as shipping all the necessary HBase
> dependencies. The other is shfty and nefarious, {{void
> addDependencyJars(Configuration, Class...)}} -- it only adds exactly what
> the user requests, forcing them to resolve dependencies themselves and giving
> a false sense of security. We should deprecate the latter throw a big giant
> warning when people use that one. The handy functionality of providing help
> when our heuristics fail can be added via a new method signature, something
> like {{void addDependencyJars(Job, Class ...}}. This method would do
> everything {{void addDependencyJars(Job}} does, plus let the user specify
> arbitrary additional classes. That way HBase still can help the user, but
> also gives them super-powers to compensate for when our heuristics fail.
> For reference, this appears to be the reason why HBase + Pig doesn't really
> work out of the box. See
> [HBaseStorage.java|https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java#L730]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16340) ensure no Xerces jars included
[ https://issues.apache.org/jira/browse/HBASE-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16340: Resolution: Fixed Release Note: HBase no longer includes Xerces implementation jars that were previously included via transitive dependencies. Downstream users relying on HBase for these artifacts will need to update their dependencies. Status: Resolved (was: Patch Available) > ensure no Xerces jars included > -- > > Key: HBASE-16340 > URL: https://issues.apache.org/jira/browse/HBASE-16340 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16340.1.patch, HBASE-16340.2.patch > > > When we moved our pom to Hadoop 2.7 we picked up a transitive Xerces > implementation. We should exclude this to ensure we don't get a conflict with > the implementation that ships with the jvm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-8386) deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
[
https://issues.apache.org/jira/browse/HBASE-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reopened HBASE-8386:
> deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
> ---
>
> Key: HBASE-8386
> URL: https://issues.apache.org/jira/browse/HBASE-8386
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-8386.1.patch
>
>
> We expose two public static methods names {{addDependencyJars}}. One of them,
> {{void addDependencyJars(Job}}, is very helpful -- goes out of its way to
> detect job dependencies as well as shipping all the necessary HBase
> dependencies. The other is shfty and nefarious, {{void
> addDependencyJars(Configuration, Class...)}} -- it only adds exactly what
> the user requests, forcing them to resolve dependencies themselves and giving
> a false sense of security. We should deprecate the latter throw a big giant
> warning when people use that one. The handy functionality of providing help
> when our heuristics fail can be added via a new method signature, something
> like {{void addDependencyJars(Job, Class ...}}. This method would do
> everything {{void addDependencyJars(Job}} does, plus let the user specify
> arbitrary additional classes. That way HBase still can help the user, but
> also gives them super-powers to compensate for when our heuristics fail.
> For reference, this appears to be the reason why HBase + Pig doesn't really
> work out of the box. See
> [HBaseStorage.java|https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java#L730]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-8386) deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
[
https://issues.apache.org/jira/browse/HBASE-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-8386:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
> ---
>
> Key: HBASE-8386
> URL: https://issues.apache.org/jira/browse/HBASE-8386
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-8386.1.patch
>
>
> We expose two public static methods names {{addDependencyJars}}. One of them,
> {{void addDependencyJars(Job}}, is very helpful -- goes out of its way to
> detect job dependencies as well as shipping all the necessary HBase
> dependencies. The other is shfty and nefarious, {{void
> addDependencyJars(Configuration, Class...)}} -- it only adds exactly what
> the user requests, forcing them to resolve dependencies themselves and giving
> a false sense of security. We should deprecate the latter throw a big giant
> warning when people use that one. The handy functionality of providing help
> when our heuristics fail can be added via a new method signature, something
> like {{void addDependencyJars(Job, Class ...}}. This method would do
> everything {{void addDependencyJars(Job}} does, plus let the user specify
> arbitrary additional classes. That way HBase still can help the user, but
> also gives them super-powers to compensate for when our heuristics fail.
> For reference, this appears to be the reason why HBase + Pig doesn't really
> work out of the box. See
> [HBaseStorage.java|https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java#L730]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-16303) FilterList with MUST_PASS_ONE optimization
[ https://issues.apache.org/jira/browse/HBASE-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-16303: - I had to revert the addendum from branch-1.3 because it broke compilation. I left the base patch in place, because TestFilter and TestFilterList passed without the addendum. Please verify that things are fine on branch-1.3 and then re-close. > FilterList with MUST_PASS_ONE optimization > -- > > Key: HBASE-16303 > URL: https://issues.apache.org/jira/browse/HBASE-16303 > Project: HBase > Issue Type: Bug > Components: Filters >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16303.patch, HBASE-16303.patch, > HBASE-16303_1.patch, HBASE-16303_1.patch > > > This JIRA was created to add @Test tag for all the filter related tests. > After doing that one such test failed in > TestFilterList#testFilterListTwoFiltersMustPassOne. > The test assumes that when a PrefixFilter is added to a filterlist once the > given prefix is passed we should SKIP all other rows. But the impl in > FilterList for MUST_PASS_ONE filter is such that once the filter in the list > sees that filterAllRemaining is true we don't use that to decide if to to > SEE_NEXT_USING_HINT or not. This patch tries to handle that case where if for > a filter FilterAllRemaining is true then SEEK_NEXT_USING_HINT need not be > done. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16361) Revert of HBASE-16317, "revert all ESAPI..." broke TestLogLevel
[ https://issues.apache.org/jira/browse/HBASE-16361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410025#comment-15410025 ] stack commented on HBASE-16361: --- No worries [~ndimiduk] Let me figure it out. It is not the revert explicitly it seems, but as you suggest, the revert reveals a hole in our dependency. Let me figure. > Revert of HBASE-16317, "revert all ESAPI..." broke TestLogLevel > --- > > Key: HBASE-16361 > URL: https://issues.apache.org/jira/browse/HBASE-16361 > Project: HBase > Issue Type: Bug > Components: dependencies, UI >Reporter: stack >Assignee: stack > > Side effect of the revert in HBASE-16317 is broken TestLogLevel. > testDynamicLogLevel(org.apache.hadoop.hbase.http.log.TestLogLevel) Time > elapsed: 0.956 sec <<< ERROR! > java.io.IOException: Server returned HTTP response code: 500 for URL: > http://localhost:51940/logLevel?log=org.apache.hadoop.hbase.http.log.TestLogLevel=ERROR > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) > at > org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:71) > Complaint is because... > 41 Caused by: java.lang.ClassNotFoundException: > org.apache.commons.httpclient.URIException > 42 at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > Let me see if I can fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16361) Revert of HBASE-16317, "revert all ESAPI..." broke TestLogLevel
[ https://issues.apache.org/jira/browse/HBASE-16361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409996#comment-15409996 ] Nick Dimiduk commented on HBASE-16361: -- [~tedyu] pointed out this regression to me; sorry about this. It looks like we have a used, undeclared dependency added with this test. > Revert of HBASE-16317, "revert all ESAPI..." broke TestLogLevel > --- > > Key: HBASE-16361 > URL: https://issues.apache.org/jira/browse/HBASE-16361 > Project: HBase > Issue Type: Bug > Components: dependencies, UI >Reporter: stack >Assignee: stack > > Side effect of the revert in HBASE-16317 is broken TestLogLevel. > testDynamicLogLevel(org.apache.hadoop.hbase.http.log.TestLogLevel) Time > elapsed: 0.956 sec <<< ERROR! > java.io.IOException: Server returned HTTP response code: 500 for URL: > http://localhost:51940/logLevel?log=org.apache.hadoop.hbase.http.log.TestLogLevel=ERROR > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) > at > org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:71) > Complaint is because... > 41 Caused by: java.lang.ClassNotFoundException: > org.apache.commons.httpclient.URIException > 42 at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > Let me see if I can fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[
https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409975#comment-15409975
]
Hadoop QA commented on HBASE-16148:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 4m 14s {color}
| {color:red} hbase-server-jdk1.7.0_80 with JDK v1.7.0_80 generated 2 new + 4
unchanged - 2 fixed = 6 total (was 6) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 47s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 15s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 59s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.http.log.TestLogLevel |
\\
\\
|| Subsystem
[jira] [Issue Comment Deleted] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Comment: was deleted (was: [~enis] Referring to the RB link, in the to do list, you have written. "Puts with client-set timestamps should update the clocks." Should we update the server clock with client set timestamps ? In HBase, client can set any timestamp he wants, it could have any skew. I feel we should not update the clock with client set timestamps, rather send another field in put request which just reads time of system clock, use that to update server clock. ) > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16321) Ensure findbugs jsr305 jar isn't present
[
https://issues.apache.org/jira/browse/HBASE-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409933#comment-15409933
]
Hudson commented on HBASE-16321:
FAILURE: Integrated in HBase-Trunk_matrix #1360 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1360/])
HBASE-16321 ensure no findbugs-jsr305 (busbey: rev
60b79e2daa41927f5a2e00b78cca855f05048f0a)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormat.java
* hbase-client/pom.xml
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContext.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
* pom.xml
*
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
* hbase-rsgroup/pom.xml
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* hbase-server/pom.xml
* hbase-spark/pom.xml
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
* src/main/asciidoc/_chapters/developer.adoc
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
> Ensure findbugs jsr305 jar isn't present
>
>
> Key: HBASE-16321
> URL: https://issues.apache.org/jira/browse/HBASE-16321
> Project: HBase
> Issue Type: Sub-task
> Components: dependencies
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: HBASE-16321.1.patch, HBASE-16321.2.patch
>
>
> we should be using
> {code}
>
>
> com.github.stephenc.findbugs
> findbugs-annotations
> ${findbugs-annotations}
> compile
>
> {code}
> to ensure we don't have a prohibited dependency, but it looks like we're
> still bringing in
> {code}
>
> com.google.code.findbugs
> jsr305
> ${jsr305.version}
>
> {code}
> remove the findbugs version (even though the maven central pom claims the
> license is ALv2, that doesn't line up with the referenced project sites).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409936#comment-15409936 ] Hudson commented on HBASE-16308: FAILURE: Integrated in HBase-Trunk_matrix #1360 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1360/]) HBASE-16308 Contain protobuf references Gather up the pb references into (stack: rev ed87a81b4b61c4842c12572a47c97ae23773012f) * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RegionCoprocessorRpcChannel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/TimeLimitedRpcController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mob/ExpiredMobFileCleaner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoadWithOldClient.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientTimeouts.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSimpleScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoadWithOldSecureEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/SecureBulkLoadClient.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/Merge.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ExpiredMobFileCleanerChore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * hbase-spark/src/main/java/org/apache/hadoop/hbase/spark/SparkSQLPushDownFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/MasterCoprocessorRpcChannel.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestNamespace.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/PayloadCarryingRpcController.java * hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientScanner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/FlushRegionCallable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionCallable.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseIOException.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingTimeTracker.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ExceptionUtil.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicaWithCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/Sweeper.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterKeepAliveConnection.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/PayloadCarryingServerCallable.java > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 >
[jira] [Commented] (HBASE-8386) deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
[
https://issues.apache.org/jira/browse/HBASE-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409937#comment-15409937
]
Hudson commented on HBASE-8386:
---
FAILURE: Integrated in HBase-Trunk_matrix #1360 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1360/])
HBASE-8386 deprecate TableMapReduce.addDependencyJars(Configuration, (busbey:
rev 431c8c9ad03336b5ab844d97adc47b8a73d7aa00)
* hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
* src/main/asciidoc/_chapters/mapreduce.adoc
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestWithCellVisibilityLoadAndVerify.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestLoadAndVerify.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestTableSnapshotInputFormat.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
> deprecate TableMapReduce.addDependencyJars(Configuration, class ...)
> ---
>
> Key: HBASE-8386
> URL: https://issues.apache.org/jira/browse/HBASE-8386
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-8386.1.patch
>
>
> We expose two public static methods names {{addDependencyJars}}. One of them,
> {{void addDependencyJars(Job}}, is very helpful -- goes out of its way to
> detect job dependencies as well as shipping all the necessary HBase
> dependencies. The other is shfty and nefarious, {{void
> addDependencyJars(Configuration, Class...)}} -- it only adds exactly what
> the user requests, forcing them to resolve dependencies themselves and giving
> a false sense of security. We should deprecate the latter throw a big giant
> warning when people use that one. The handy functionality of providing help
> when our heuristics fail can be added via a new method signature, something
> like {{void addDependencyJars(Job, Class ...}}. This method would do
> everything {{void addDependencyJars(Job}} does, plus let the user specify
> arbitrary additional classes. That way HBase still can help the user, but
> also gives them super-powers to compensate for when our heuristics fail.
> For reference, this appears to be the reason why HBase + Pig doesn't really
> work out of the box. See
> [HBaseStorage.java|https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java#L730]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16340) ensure no Xerces jars included
[ https://issues.apache.org/jira/browse/HBASE-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409935#comment-15409935 ] Hudson commented on HBASE-16340: FAILURE: Integrated in HBase-Trunk_matrix #1360 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1360/]) HBASE-16340 exclude Xerces iplementation jars from coming in (busbey: rev c59f76485e2fbb86124847c4b31a06dc53d393b8) * pom.xml * hbase-spark/pom.xml > ensure no Xerces jars included > -- > > Key: HBASE-16340 > URL: https://issues.apache.org/jira/browse/HBASE-16340 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16340.1.patch, HBASE-16340.2.patch > > > When we moved our pom to Hadoop 2.7 we picked up a transitive Xerces > implementation. We should exclude this to ensure we don't get a conflict with > the implementation that ships with the jvm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15461) ref guide has bad links to blogs originally posted on cloudera website
[ https://issues.apache.org/jira/browse/HBASE-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409934#comment-15409934 ] Hudson commented on HBASE-15461: FAILURE: Integrated in HBase-Trunk_matrix #1360 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1360/]) HBASE-15461 update links to blogs posted on the Cloudera website to (busbey: rev 48af3f5f7a38fe70b3518a700ccd804c2aac022d) * src/main/asciidoc/_chapters/security.adoc * src/main/asciidoc/_chapters/configuration.adoc * src/main/asciidoc/_chapters/other_info.adoc * src/main/asciidoc/_chapters/troubleshooting.adoc * src/main/asciidoc/_chapters/ops_mgt.adoc > ref guide has bad links to blogs originally posted on cloudera website > -- > > Key: HBASE-15461 > URL: https://issues.apache.org/jira/browse/HBASE-15461 > Project: HBase > Issue Type: Bug > Components: website >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0 > > Attachments: HBASE-15461.1.patch, HBASE-15461.2.patch > > > The ref guide section on "Secure Client Access to Apache HBase" starts with a > link to a blog post from Matteo, but the link is broken. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15867) Move HBase replication tracking from ZooKeeper to HBase
[ https://issues.apache.org/jira/browse/HBASE-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15867: --- Assignee: Ashu Pachauri (was: Joseph) > Move HBase replication tracking from ZooKeeper to HBase > --- > > Key: HBASE-15867 > URL: https://issues.apache.org/jira/browse/HBASE-15867 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri > > Move the WAL file and offset tracking out of ZooKeeper and into an HBase > table called hbase:replication. > The largest three new changes will be two classes ReplicationTableBase, > TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now > ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks > have been filed for these two jobs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16138: --- Assignee: Ashu Pachauri (was: Joseph) > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri >Priority: Critical > Attachments: HBASE-16138-v1.patch, HBASE-16138.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16342) Backport replication tracking via system table to Branch-1
[ https://issues.apache.org/jira/browse/HBASE-16342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16342: --- Assignee: Ashu Pachauri (was: Joseph) > Backport replication tracking via system table to Branch-1 > -- > > Key: HBASE-16342 > URL: https://issues.apache.org/jira/browse/HBASE-16342 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri > Attachments: HBASE-16342-branch-1-v2.patch, HBASE-16342-branch-1.patch > > > HBASE-16342 Added Replication WAL offset tracking out of ZooKeeper and into > an HBase table. Provided new implementation of ReplicationQueues and > ReplicationQueuesClient that implement this. Peer tracking still remains > inside of ZooKeeper though. Includes H-Base issues: HBASE-15883, HBASE-15958, > HBASE-15974, HBASE-16036, HBASE-16083, HBASE-16080, HBASE-1620, and > HBASE-15937 > Tested these changes on Branch-1 with IntegrationTestReplication and it > appears stable. Can also handle late master startups and replication table > regions going down as long as only system tables are hosted on Master and > hbase:replication is hosted on Master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15937) Figure out retry limit and timing for replication queue table operations
[ https://issues.apache.org/jira/browse/HBASE-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15937: --- Assignee: Ashu Pachauri (was: Joseph) > Figure out retry limit and timing for replication queue table operations > > > Key: HBASE-15937 > URL: https://issues.apache.org/jira/browse/HBASE-15937 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Ashu Pachauri > Attachments: HBASE-15937.patch > > > ReplicationQueuesHBaseImpl will abort the server if any of its HBase Table > writes/reads fails. We should figure out a reasonable retry limit and pause > duration for these operations. > As of now the timeouts look like: > Table initialization: > 240 retries > 1 minute pause (because the Master may not be initialized yet, createTable > retries are immediately rejected by PleaseHoldException, so we should sleep > in between RPC requests) > 1 minute RPC timeouts > Total: At minimum 2 hours of retries > Normal Replication Table operations: > 240 retries > 100 millis pause (because we assume the cluster is in a more stable state, we > assume most exceptions will be RPC timeouts, so I am using the standard RPC > pause) > 1 minute RPC timeouts > Total: Assuming operations fail because of RPC timeouts, a minimum of 2 hours > of retries. With just pauses we only have 24 seconds. > All of these timeouts are configurable too though. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16209) Provide an ExponentialBackOffPolicy sleep between failed region open requests
[ https://issues.apache.org/jira/browse/HBASE-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16209: --- Assignee: Ashu Pachauri (was: Joseph) > Provide an ExponentialBackOffPolicy sleep between failed region open requests > - > > Key: HBASE-16209 > URL: https://issues.apache.org/jira/browse/HBASE-16209 > Project: HBase > Issue Type: Bug >Reporter: Joseph >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16209-addendum.patch, > HBASE-16209-branch-1-addendum-v2.patch, HBASE-16209-branch-1-addendum.patch, > HBASE-16209-branch-1-v3.patch, HBASE-16209-branch-1-v4.patch, > HBASE-16209-branch-1-v5.patch, HBASE-16209-branch-1.patch, > HBASE-16209-v2.patch, HBASE-16209.patch > > > Related to HBASE-16138. As of now we currently have no pause between retrying > failed region open requests. And with a low maximumAttempt default, we can > quickly use up all our regionOpen retries if the server is in a bad state. I > added in a ExponentialBackOffPolicy so that we spread out the timing of our > open region retries in AssignmentManager. Review board at > https://reviews.apache.org/r/50011/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16138: --- Status: Patch Available (was: Open) > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph >Priority: Critical > Attachments: HBASE-16138-v1.patch, HBASE-16138.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16138: --- Attachment: HBASE-16138-v1.patch Made changes requested by Gary > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph >Priority: Critical > Attachments: HBASE-16138-v1.patch, HBASE-16138.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16138) Cannot open regions after non-graceful shutdown due to deadlock with Replication Table
[ https://issues.apache.org/jira/browse/HBASE-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16138: --- Status: Open (was: Patch Available) > Cannot open regions after non-graceful shutdown due to deadlock with > Replication Table > -- > > Key: HBASE-16138 > URL: https://issues.apache.org/jira/browse/HBASE-16138 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph >Priority: Critical > Attachments: HBASE-16138-v1.patch, HBASE-16138.patch > > > If we shutdown an entire HBase cluster and attempt to start it back up, we > have to run the WAL pre-log roll that occurs before opening up a region. Yet > this pre-log roll must record the new WAL inside of ReplicationQueues. This > method call ends up blocking on > TableBasedReplicationQueues.getOrBlockOnReplicationTable(), because the > Replication Table is not up yet. And we cannot assign the Replication Table > because we cannot open any regions. This ends up deadlocking the entire > cluster whenever we lose Replication Table availability. > There are a few options that we can do, but none of them seem very good: > 1. Depend on Zookeeper-based Replication until the Replication Table becomes > available > 2. Have a separate WAL for System Tables that does not perform any > replication (see discussion at HBASE-14623) > Or just have a seperate WAL for non-replicated vs replicated > regions > 3. Record the WAL log in the ReplicationQueue asynchronously (don't block > opening a region on this event), which could lead to inconsistent Replication > state > The stacktrace: > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.recordLog(ReplicationSourceManager.java:376) > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.preLogRoll(ReplicationSourceManager.java:348) > > org.apache.hadoop.hbase.replication.regionserver.Replication.preLogRoll(Replication.java:370) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.tellListenersAboutPreLogRoll(FSHLog.java:637) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:600) > > org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:533) > > org.apache.hadoop.hbase.wal.DefaultWALProvider.getWAL(DefaultWALProvider.java:132) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:186) > > org.apache.hadoop.hbase.wal.RegionGroupingProvider.getWAL(RegionGroupingProvider.java:197) > org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:240) > > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:1883) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Does anyone have any suggestions/ideas/feedback? > Attached a review board at: https://reviews.apache.org/r/50546/ > It is still pretty rough, would just like some feedback on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[
https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409928#comment-15409928
]
Hadoop QA commented on HBASE-15866:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15866 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822357/read-write-rpc-timeouts-edited.v2.patch
|
| JIRA Issue | HBASE-15866 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2966/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
> ---
>
> Key: HBASE-15866
> URL: https://issues.apache.org/jira/browse/HBASE-15866
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Andrew Purtell
>Assignee: Vivek Koppuru
> Labels: newbie, patch
> Fix For: 2.0.0
>
> Attachments: HBASE-15866.patch,
> read-write-rpc-timeouts-edited.v2.patch, read-write-rpc-timeouts.patch
>
>
> We have a single tunable for the RPC timeout interval - hbase.rpc.timeout.
> This is fine for the general case but there are use cases where it would be
> advantageous to set two separate timeouts for reads (gets, scans, perhaps
> with significant server side filtering - although the new scanner heartbeat
> feature mitigates where available) and mutations (fail fast under tight SLA,
> resubmit or take mitigating action).
> I propose we refer to a configuration setting "hbase.rpc.read.timeout" when
> handling read operations and "hbase.rpc.write.timeout" when handling write
> operations. If those values are not set in the configuration, fall back to
> the value of "hbase.rpc.timeout" or its default.
> So for example in HTable instead of one global timeout for each RPC
> (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up
> in HTable#finishSetup. Then wherever we set up RPC with
> RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or
> write timeout depending on what the op is.
> In general I don't like the idea of adding configuration parameters to our
> already heavyweight set, but I think the inability to control timeouts
> separately for reads and writes is an operational deficit.
> See also PHOENIX-2916.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16342) Backport replication tracking via system table to Branch-1
[ https://issues.apache.org/jira/browse/HBASE-16342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16342: --- Status: Patch Available (was: Open) > Backport replication tracking via system table to Branch-1 > -- > > Key: HBASE-16342 > URL: https://issues.apache.org/jira/browse/HBASE-16342 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16342-branch-1-v2.patch, HBASE-16342-branch-1.patch > > > HBASE-16342 Added Replication WAL offset tracking out of ZooKeeper and into > an HBase table. Provided new implementation of ReplicationQueues and > ReplicationQueuesClient that implement this. Peer tracking still remains > inside of ZooKeeper though. Includes H-Base issues: HBASE-15883, HBASE-15958, > HBASE-15974, HBASE-16036, HBASE-16083, HBASE-16080, HBASE-1620, and > HBASE-15937 > Tested these changes on Branch-1 with IntegrationTestReplication and it > appears stable. Can also handle late master startups and replication table > regions going down as long as only system tables are hosted on Master and > hbase:replication is hosted on Master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16342) Backport replication tracking via system table to Branch-1
[ https://issues.apache.org/jira/browse/HBASE-16342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16342: --- Attachment: HBASE-16342-branch-1-v2.patch Added in UI description of the Replication Table > Backport replication tracking via system table to Branch-1 > -- > > Key: HBASE-16342 > URL: https://issues.apache.org/jira/browse/HBASE-16342 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16342-branch-1-v2.patch, HBASE-16342-branch-1.patch > > > HBASE-16342 Added Replication WAL offset tracking out of ZooKeeper and into > an HBase table. Provided new implementation of ReplicationQueues and > ReplicationQueuesClient that implement this. Peer tracking still remains > inside of ZooKeeper though. Includes H-Base issues: HBASE-15883, HBASE-15958, > HBASE-15974, HBASE-16036, HBASE-16083, HBASE-16080, HBASE-1620, and > HBASE-15937 > Tested these changes on Branch-1 with IntegrationTestReplication and it > appears stable. Can also handle late master startups and replication table > regions going down as long as only system tables are hosted on Master and > hbase:replication is hosted on Master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16342) Backport replication tracking via system table to Branch-1
[ https://issues.apache.org/jira/browse/HBASE-16342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16342: --- Status: Open (was: Patch Available) > Backport replication tracking via system table to Branch-1 > -- > > Key: HBASE-16342 > URL: https://issues.apache.org/jira/browse/HBASE-16342 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-16342-branch-1.patch > > > HBASE-16342 Added Replication WAL offset tracking out of ZooKeeper and into > an HBase table. Provided new implementation of ReplicationQueues and > ReplicationQueuesClient that implement this. Peer tracking still remains > inside of ZooKeeper though. Includes H-Base issues: HBASE-15883, HBASE-15958, > HBASE-15974, HBASE-16036, HBASE-16083, HBASE-16080, HBASE-1620, and > HBASE-15937 > Tested these changes on Branch-1 with IntegrationTestReplication and it > appears stable. Can also handle late master startups and replication table > regions going down as long as only system tables are hosted on Master and > hbase:replication is hosted on Master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Koppuru updated HBASE-15866: -- Release Note: Fixed broken unit test and added unit tests for TestHCM#testWriteRpcTimeout and TestHCM#testReadRpcTimeout, along with necessary changes to make that work. This is the patch labelled "read-write-rpc-timeouts-edited.v2.patch" Status: Patch Available (was: In Progress) > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, > read-write-rpc-timeouts-edited.v2.patch, read-write-rpc-timeouts.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15866) Split hbase.rpc.timeout into *.read.timeout and *.write.timeout
[ https://issues.apache.org/jira/browse/HBASE-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Koppuru updated HBASE-15866: -- Attachment: read-write-rpc-timeouts-edited.v2.patch > Split hbase.rpc.timeout into *.read.timeout and *.write.timeout > --- > > Key: HBASE-15866 > URL: https://issues.apache.org/jira/browse/HBASE-15866 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Vivek Koppuru > Labels: newbie, patch > Fix For: 2.0.0 > > Attachments: HBASE-15866.patch, > read-write-rpc-timeouts-edited.v2.patch, read-write-rpc-timeouts.patch > > > We have a single tunable for the RPC timeout interval - hbase.rpc.timeout. > This is fine for the general case but there are use cases where it would be > advantageous to set two separate timeouts for reads (gets, scans, perhaps > with significant server side filtering - although the new scanner heartbeat > feature mitigates where available) and mutations (fail fast under tight SLA, > resubmit or take mitigating action). > I propose we refer to a configuration setting "hbase.rpc.read.timeout" when > handling read operations and "hbase.rpc.write.timeout" when handling write > operations. If those values are not set in the configuration, fall back to > the value of "hbase.rpc.timeout" or its default. > So for example in HTable instead of one global timeout for each RPC > (rpcTimeout), there would be a readRpcTimeout and writeRpcTimeout also set up > in HTable#finishSetup. Then wherever we set up RPC with > RpcRetryingCallerFactory#newCaller(int rpcTimeout) we pass in the read or > write timeout depending on what the op is. > In general I don't like the idea of adding configuration parameters to our > already heavyweight set, but I think the inability to control timeouts > separately for reads and writes is an operational deficit. > See also PHOENIX-2916. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Status: Open (was: Patch Available) > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Status: Patch Available (was: Open) > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Attachment: HLC.2.patch > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16310) Revisit the logic of filterRowKey for Filters
[ https://issues.apache.org/jira/browse/HBASE-16310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409875#comment-15409875 ] Sean Busbey commented on HBASE-16310: - precommit only runs unit tests in the modules that a given patch changes. if you want test to run in additional modules, you'll have to cause a change there. an easy way to test project-wide is to include a second commit that touches the top level pom. Just don't push that second commit at the end. > Revisit the logic of filterRowKey for Filters > - > > Key: HBASE-16310 > URL: https://issues.apache.org/jira/browse/HBASE-16310 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.3, 1.1.5, 1.2.2 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16310.patch, HBASE-16310_1.patch, > HBASE-16310_1.patch, HBASE-16310_1.patch > > > Based on HBASE-16926 it was observed that filterRowKey() API behaviour for > FilterList and a standalone filter should be made consistent. This JIRA is to > track down the changes and ensure 1.3+ on wards we have consistent behaviour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16198) Enhance backup history command
[
https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409869#comment-15409869
]
Ted Yu commented on HBASE-16198:
In patch v4, I still see:
{code}
+private Path getBackupRootPath() {
+ String value = cmdline.getOptionValue("t");
{code}
Is there any change in BackupManifest.java ?
> Enhance backup history command
> --
>
> Key: HBASE-16198
> URL: https://issues.apache.org/jira/browse/HBASE-16198
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16198-v1.patch, HBASE-16198-v2.patch,
> HBASE-16198-v3.patch, HBASE-16198-v4.patch
>
>
> We need history per table and ability to run command on a fully disabled
> cluster (using info from backup file system only).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409853#comment-15409853 ] Sai Teja Ranuva commented on HBASE-16148: - [~enis] Referring to the RB link, in the to do list, you have written. "Puts with client-set timestamps should update the clocks." Should we update the server clock with client set timestamps ? In HBase, client can set any timestamp he wants, it could have any skew. I feel we should not update the clock with client set timestamps, rather send another field in put request which just reads time of system clock, use that to update server clock. > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16303) FilterList with MUST_PASS_ONE optimization
[
https://issues.apache.org/jira/browse/HBASE-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409854#comment-15409854
]
Sean Busbey commented on HBASE-16303:
-
{quote}
bq. unnecessary results retrieved, but not returned incorrectly? what happens
without the fix, incorrect results?
In the actual case I don't think it will produce wrong results, but may be
unnecessary iterations. I can verify that if needed.
{quote}
Thanks! returning correct results or not is how I'd differentiate between this
being an improvement (and thus only needed on 1.3+) vs a bug (and thus needed
on all branches).
> FilterList with MUST_PASS_ONE optimization
> --
>
> Key: HBASE-16303
> URL: https://issues.apache.org/jira/browse/HBASE-16303
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16303.patch, HBASE-16303.patch,
> HBASE-16303_1.patch, HBASE-16303_1.patch
>
>
> This JIRA was created to add @Test tag for all the filter related tests.
> After doing that one such test failed in
> TestFilterList#testFilterListTwoFiltersMustPassOne.
> The test assumes that when a PrefixFilter is added to a filterlist once the
> given prefix is passed we should SKIP all other rows. But the impl in
> FilterList for MUST_PASS_ONE filter is such that once the filter in the list
> sees that filterAllRemaining is true we don't use that to decide if to to
> SEE_NEXT_USING_HINT or not. This patch tries to handle that case where if for
> a filter FilterAllRemaining is true then SEEK_NEXT_USING_HINT need not be
> done.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16364) Enhance BackupSystemTable#getBackupHistoryForTable() to selectively retrieve related backup history
[ https://issues.apache.org/jira/browse/HBASE-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16364: --- Labels: HBASE-16198 (was: ) > Enhance BackupSystemTable#getBackupHistoryForTable() to selectively retrieve > related backup history > --- > > Key: HBASE-16364 > URL: https://issues.apache.org/jira/browse/HBASE-16364 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > Labels: backup > > During review of HBASE-16198, it was noticed that > BackupSystemTable#getBackupHistoryForTable() has to first retrieve all backup > history and then does the filtering. > For a cluster where there has been a lot of backup activity, the amount of > data retrieved may be non-trivial. > This issue is to enhance BackupSystemTable#getBackupHistoryForTable() so that > filtering is applied in the first place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16364) Enhance BackupSystemTable#getBackupHistoryForTable() to selectively retrieve related backup history
[ https://issues.apache.org/jira/browse/HBASE-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16364: --- Labels: backup (was: HBASE-16198) > Enhance BackupSystemTable#getBackupHistoryForTable() to selectively retrieve > related backup history > --- > > Key: HBASE-16364 > URL: https://issues.apache.org/jira/browse/HBASE-16364 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > Labels: backup > > During review of HBASE-16198, it was noticed that > BackupSystemTable#getBackupHistoryForTable() has to first retrieve all backup > history and then does the filtering. > For a cluster where there has been a lot of backup activity, the amount of > data retrieved may be non-trivial. > This issue is to enhance BackupSystemTable#getBackupHistoryForTable() so that > filtering is applied in the first place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16364) Enhance BackupSystemTable#getBackupHistoryForTable() to selectively retrieve related backup history
Ted Yu created HBASE-16364: -- Summary: Enhance BackupSystemTable#getBackupHistoryForTable() to selectively retrieve related backup history Key: HBASE-16364 URL: https://issues.apache.org/jira/browse/HBASE-16364 Project: HBase Issue Type: Improvement Reporter: Ted Yu During review of HBASE-16198, it was noticed that BackupSystemTable#getBackupHistoryForTable() has to first retrieve all backup history and then does the filtering. For a cluster where there has been a lot of backup activity, the amount of data retrieved may be non-trivial. This issue is to enhance BackupSystemTable#getBackupHistoryForTable() so that filtering is applied in the first place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16340) ensure no Xerces jars included
[ https://issues.apache.org/jira/browse/HBASE-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409844#comment-15409844 ] Sean Busbey commented on HBASE-16340: - arg. already pushed. I'll add your signoff to the back port branches. sorry! > ensure no Xerces jars included > -- > > Key: HBASE-16340 > URL: https://issues.apache.org/jira/browse/HBASE-16340 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16340.1.patch, HBASE-16340.2.patch > > > When we moved our pom to Hadoop 2.7 we picked up a transitive Xerces > implementation. We should exclude this to ensure we don't get a conflict with > the implementation that ships with the jvm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16358) Backup create command: log success/failure with backup id
[ https://issues.apache.org/jira/browse/HBASE-16358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16358: --- Labels: backup (was: ) > Backup create command: log success/failure with backup id > - > > Key: HBASE-16358 > URL: https://issues.apache.org/jira/browse/HBASE-16358 > Project: HBase > Issue Type: Improvement >Affects Versions: HBASE-7912 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Labels: backup > Fix For: HBASE-7912 > > Attachments: HBASE-16358-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16358) Backup create command: log success/failure with backup id
[ https://issues.apache.org/jira/browse/HBASE-16358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-16358. Resolution: Fixed > Backup create command: log success/failure with backup id > - > > Key: HBASE-16358 > URL: https://issues.apache.org/jira/browse/HBASE-16358 > Project: HBase > Issue Type: Improvement >Affects Versions: HBASE-7912 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Labels: backup > Fix For: HBASE-7912 > > Attachments: HBASE-16358-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
