[jira] [Commented] (HBASE-7424) Enable the DeltaEncoding for the HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600898#comment-13600898 ] Anoop Sam John commented on HBASE-7424: --- Okey. Any way Manukranth Kolloju resolved this issue and its version is 0.89fb > Enable the DeltaEncoding for the HFileOutputFormat > -- > > Key: HBASE-7424 > URL: https://issues.apache.org/jira/browse/HBASE-7424 > Project: HBase > Issue Type: New Feature >Reporter: Manukranth Kolloju >Priority: Minor > Labels: HFileOutputFormat > Fix For: 0.89-fb > > > HFileOutputFormat has a writer embedded but it is not configured to use the > DeltaEncoding. This revision is to add that support to the HFileOutputFormat > while it is used as an OutputFormat either the Mapper or the Reducer for a > MapReduce task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7403) Online Merge
[ https://issues.apache.org/jira/browse/HBASE-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-7403: Attachment: hbase-7403-trunkv22.patch Addressing latest comments from review board > Online Merge > > > Key: HBASE-7403 > URL: https://issues.apache.org/jira/browse/HBASE-7403 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.95.0 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7403-trunkv5.patch, 7403-trunkv6.patch, 7403v5.diff, > 7403-v5.txt, 7403v5.txt, hbase-7403-94v1.patch, hbase-7403-trunkv10.patch, > hbase-7403-trunkv11.patch, hbase-7403-trunkv12.patch, > hbase-7403-trunkv13.patch, hbase-7403-trunkv14.patch, > hbase-7403-trunkv15.patch, hbase-7403-trunkv16.patch, > hbase-7403-trunkv19.patch, hbase-7403-trunkv1.patch, > hbase-7403-trunkv20.patch, hbase-7403-trunkv22.patch, > hbase-7403-trunkv5.patch, hbase-7403-trunkv6.patch, hbase-7403-trunkv7.patch, > hbase-7403-trunkv8.patch, hbase-7403-trunkv9.patch, merge region.pdf > > > Support executing region merge transaction on Regionserver, similar with > split transaction > Process of merging two regions: > a.client send RPC(dispacth merging regions) to master > b.master move the regions together (on the same regionserver) > c.master send RPC(merge regions) to regionserver > d.Regionserver execute the regions merge transaction in the thread pool > e.the above b,c,d run asynchronously > Process of region merge transaction: > a.Construct a new region merge transaction. > b.prepare for the merge transaction, the transaction will be canceled if it > is unavailable, > e.g. two regions don't belong to same table; two regions are not adjacent in > a non-compulsory merge; region is closed or has reference > c.execute the transaction as the following: > /** > * Set region as in transition, set it into MERGING state. > */ > SET_MERGING_IN_ZK, > /** > * We created the temporary merge data directory. > */ > CREATED_MERGE_DIR, > /** > * Closed the merging region A. > */ > CLOSED_REGION_A, > /** > * The merging region A has been taken out of the server's online regions > list. > */ > OFFLINED_REGION_A, > /** > * Closed the merging region B. > */ > CLOSED_REGION_B, > /** > * The merging region B has been taken out of the server's online regions > list. > */ > OFFLINED_REGION_B, > /** > * Started in on creation of the merged region. > */ > STARTED_MERGED_REGION_CREATION, > /** > * Point of no return. If we got here, then transaction is not recoverable > * other than by crashing out the regionserver. > */ > PONR > d.roll back if step c throws exception > Usage: > HBaseAdmin#mergeRegions > See more details from the patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7952) Remove update() and Improve ExplicitColumnTracker performance.
[ https://issues.apache.org/jira/browse/HBASE-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600939#comment-13600939 ] Hudson commented on HBASE-7952: --- Integrated in hbase-0.95 #68 (See [https://builds.apache.org/job/hbase-0.95/68/]) HBASE-7952 Remove update() and Improve ExplicitColumnTracker performance (Revision 1455804) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java > Remove update() and Improve ExplicitColumnTracker performance. > -- > > Key: HBASE-7952 > URL: https://issues.apache.org/jira/browse/HBASE-7952 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.1, 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE_7952.patch, HBASE_7952_v2.patch, > HBASE_7952_v3.patch > > > In ColumnTracker.java, the update() method is not used by anyone now. And no > one will call checkColumn for different HFiles with update() in between files > to re-walk through the target columns. All columns will be feed to > checkColumn() in order. > So, within ExplicitColumnTracker, the target columns can be optimized to not > dynamic maintain a changing list of columns yet to match. Instead, just move > index through it is enough. > with this optimization to save the time for avoid reconstruct a columns array > upon each row, the checkColumn method's performance could be improved by > 10-20%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7403) Online Merge
[
https://issues.apache.org/jira/browse/HBASE-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600958#comment-13600958
]
Hadoop QA commented on HBASE-7403:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573486/hbase-7403-trunkv22.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4796//console
This message is automatically generated.
> Online Merge
>
>
> Key: HBASE-7403
> URL: https://issues.apache.org/jira/browse/HBASE-7403
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 0.95.0
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.95.0, 0.98.0
>
> Attachments: 7403-trunkv5.patch, 7403-trunkv6.patch, 7403v5.diff,
> 7403-v5.txt, 7403v5.txt, hbase-7403-94v1.patch, hbase-7403-trunkv10.patch,
> hbase-7403-trunkv11.patch, hbase-7403-trunkv12.patch,
> hbase-7403-trunkv13.patch, hbase-7403-trunkv14.patch,
> hbase-7403-trunkv15.patch, hbase-7403-trunkv16.patch,
> hbase-7403-trunkv19.patch, hbase-7403-trunkv1.patch,
> hbase-7403-trunkv20.patch, hbase-7403-trunkv22.patch,
> hbase-7403-trunkv5.patch, hbase-7403-trunkv6.patch, hbase-7403-trunkv7.patch,
> hbase-7403-trunkv8.patch, hbase-7403-trunkv9.patch, merge region.pdf
>
>
> Support executing region merge transaction on Regionserver, similar with
> split transaction
> Process of merging two regions:
> a.client send RPC(dispacth merging regions) to master
> b.master move the regions together (on the same regionserver)
> c.master send RPC(merge regions) to regionserver
> d.Regionserver execute the regions merge transaction in the thread pool
> e.the above b,c,d run asynchronously
> Process of region merge transaction:
> a.Construct a new region merge transaction.
> b.prepare for the merge transaction, the transaction will be canceled if it
> is unavailable,
> e.g. two regions don't belong to same table; two regions are not adjacent in
> a non-compulsory merge; region is closed or has reference
> c.execute the transaction as the following:
> /**
> * Set region as in transition, set it into MERGING state.
> */
> SET_MERGING_IN_ZK,
> /**
> * We created the temporary merge data directory.
> */
> CREATED_MERGE_DIR,
> /**
> * Closed the merging region A.
> */
> CLOSED_REGION_A,
> /**
> * The merging region A has been taken out of the server's online regions
> list.
>
[jira] [Commented] (HBASE-8081) Backport HBASE-7213 (separate hlog for meta tables) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600969#comment-13600969 ] nkeywal commented on HBASE-8081: Added to this, there are a lot of critical scenarios that you can have without separated logs: - some blocks in the WAL may be not recoverable (corrupted, too many boxes missing). This risk is highly mitigated with a separate log. Without this, the whole cluster becomes unavailable when you're unlucky. - if you come into hdfs issues during the recovery (hdfs issue being going to a dead datanode, something highly probable during a recovery), the recovery will be much slower. - trying to run a recovery while .meta. is not available is also problematic. Unsuring that .meta. comes back early simplifies a lot of critical scenarios. So having this in 0.94 is 'interesting' I would say :-). > Backport HBASE-7213 (separate hlog for meta tables) to 0.94 > --- > > Key: HBASE-8081 > URL: https://issues.apache.org/jira/browse/HBASE-8081 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.5 >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.94.7 > > Attachments: 7213-0.94-2.patch, 7213-0.94.patch > > > I am interested in backporting HBASE-7213 to 0.94. Helps to address more of > the MTTR story. Offline discussion with Lars indicated he is interested as > well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7403) Online Merge
[ https://issues.apache.org/jira/browse/HBASE-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600986#comment-13600986 ] chunhui shen commented on HBASE-7403: - This could be ready to go. Anybody to take a look? > Online Merge > > > Key: HBASE-7403 > URL: https://issues.apache.org/jira/browse/HBASE-7403 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.95.0 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7403-trunkv5.patch, 7403-trunkv6.patch, 7403v5.diff, > 7403-v5.txt, 7403v5.txt, hbase-7403-94v1.patch, hbase-7403-trunkv10.patch, > hbase-7403-trunkv11.patch, hbase-7403-trunkv12.patch, > hbase-7403-trunkv13.patch, hbase-7403-trunkv14.patch, > hbase-7403-trunkv15.patch, hbase-7403-trunkv16.patch, > hbase-7403-trunkv19.patch, hbase-7403-trunkv1.patch, > hbase-7403-trunkv20.patch, hbase-7403-trunkv22.patch, > hbase-7403-trunkv5.patch, hbase-7403-trunkv6.patch, hbase-7403-trunkv7.patch, > hbase-7403-trunkv8.patch, hbase-7403-trunkv9.patch, merge region.pdf > > > Support executing region merge transaction on Regionserver, similar with > split transaction > Process of merging two regions: > a.client send RPC(dispacth merging regions) to master > b.master move the regions together (on the same regionserver) > c.master send RPC(merge regions) to regionserver > d.Regionserver execute the regions merge transaction in the thread pool > e.the above b,c,d run asynchronously > Process of region merge transaction: > a.Construct a new region merge transaction. > b.prepare for the merge transaction, the transaction will be canceled if it > is unavailable, > e.g. two regions don't belong to same table; two regions are not adjacent in > a non-compulsory merge; region is closed or has reference > c.execute the transaction as the following: > /** > * Set region as in transition, set it into MERGING state. > */ > SET_MERGING_IN_ZK, > /** > * We created the temporary merge data directory. > */ > CREATED_MERGE_DIR, > /** > * Closed the merging region A. > */ > CLOSED_REGION_A, > /** > * The merging region A has been taken out of the server's online regions > list. > */ > OFFLINED_REGION_A, > /** > * Closed the merging region B. > */ > CLOSED_REGION_B, > /** > * The merging region B has been taken out of the server's online regions > list. > */ > OFFLINED_REGION_B, > /** > * Started in on creation of the merged region. > */ > STARTED_MERGED_REGION_CREATION, > /** > * Point of no return. If we got here, then transaction is not recoverable > * other than by crashing out the regionserver. > */ > PONR > d.roll back if step c throws exception > Usage: > HBaseAdmin#mergeRegions > See more details from the patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8087) Add result information in ProtoBuf response object for master/rs ops
rajeshbabu created HBASE-8087: - Summary: Add result information in ProtoBuf response object for master/rs ops Key: HBASE-8087 URL: https://issues.apache.org/jira/browse/HBASE-8087 Project: HBase Issue Type: Improvement Reporter: rajeshbabu Assignee: rajeshbabu Priority: Minor Pls refer to the comments https://issues.apache.org/jira/browse/HBASE-7992?focusedCommentId=13600513&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13600513 https://issues.apache.org/jira/browse/HBASE-7992?focusedCommentId=13600780&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13600780 Raised this issue to solve that comments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600992#comment-13600992 ] rajeshbabu commented on HBASE-7992: --- [~andrew.purt...@gmail.com] Thanks for review. bq. Should the CP be able to modify the response? Should there be a flag in the response that says the action was not taken? It may be big change, As Anoop suggested raised another JIRA(HBASE-8087) to solve this and I will work on that. > provide pre/post region offline hooks for HMaster.offlineRegion(). > --- > > Key: HBASE-7992 > URL: https://issues.apache.org/jira/browse/HBASE-7992 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Affects Versions: 0.95.0 >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.0 > > Attachments: HBASE-7992_trunk.patch > > > presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8001) Avoid unnecessary lazy seek
[ https://issues.apache.org/jira/browse/HBASE-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601002#comment-13601002 ] Ted Yu commented on HBASE-8001: --- @raymond: Can you take a look at the test failure above ? Thanks > Avoid unnecessary lazy seek > --- > > Key: HBASE-8001 > URL: https://issues.apache.org/jira/browse/HBASE-8001 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu > Fix For: 0.98.0 > > Attachments: HBASE-8001_onescanner.patch, > HBASE-8001_onescanner_v2.patch > > > Lazy seek helps to reduce the real seek needed for multi hfile, when the kv > from newer hfile is enough to satisfy the query. > While in many case, it just push the real seek later, and do not reduce the > number of real seek. e.g. there are only one hfile, or storefilescanner is > closed and only one left, or the scan need to go through all the versions, or > there are only one version of row and a sequence scan is performed. In these > case, lazy seek just bring extra overhead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601085#comment-13601085
]
Hudson commented on HBASE-7442:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #445 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/445/])
HBASE-7482 Port HBASE-7442 HBase remote CopyTable not working when security
enabled to trunk (Revision 1455764)
Result = FAILURE
garyh :
Files :
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/ProtobufRpcClientEngine.java
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/RandomTimeoutRpcEngine.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestProtoBufRpc.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClusterId.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
>Assignee: James Kinley
> Fix For: 0.92.3, 0.94.4
>
> Attachments: attempt_201212271546_0001_m_00_0.log,
> HBASE-7442-0.92.1.patch, HBASE-7442-0.94.patch
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601086#comment-13601086
]
Hudson commented on HBASE-7482:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #445 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/445/])
HBASE-7482 Port HBASE-7442 HBase remote CopyTable not working when security
enabled to trunk (Revision 1455764)
Result = FAILURE
garyh :
Files :
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/ProtobufRpcClientEngine.java
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/RandomTimeoutRpcEngine.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestProtoBufRpc.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClusterId.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch, HBASE-7482-v2.patch,
> HBASE-7482-v3.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7952) Remove update() and Improve ExplicitColumnTracker performance.
[ https://issues.apache.org/jira/browse/HBASE-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601123#comment-13601123 ] Hudson commented on HBASE-7952: --- Integrated in hbase-0.95-on-hadoop2 #24 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/24/]) HBASE-7952 Remove update() and Improve ExplicitColumnTracker performance (Revision 1455804) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java > Remove update() and Improve ExplicitColumnTracker performance. > -- > > Key: HBASE-7952 > URL: https://issues.apache.org/jira/browse/HBASE-7952 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.1, 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE_7952.patch, HBASE_7952_v2.patch, > HBASE_7952_v3.patch > > > In ColumnTracker.java, the update() method is not used by anyone now. And no > one will call checkColumn for different HFiles with update() in between files > to re-walk through the target columns. All columns will be feed to > checkColumn() in order. > So, within ExplicitColumnTracker, the target columns can be optimized to not > dynamic maintain a changing list of columns yet to match. Instead, just move > index through it is enough. > with this optimization to save the time for avoid reconstruct a columns array > upon each row, the checkColumn method's performance could be improved by > 10-20%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601124#comment-13601124
]
Hudson commented on HBASE-7482:
---
Integrated in hbase-0.95-on-hadoop2 #24 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/24/])
HBASE-7482 Port HBASE-7442 HBase remote CopyTable not working when security
enabled to trunk (Revision 1455765)
Result = FAILURE
garyh :
Files :
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/ProtobufRpcClientEngine.java
*
/hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/RandomTimeoutRpcEngine.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestProtoBufRpc.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClusterId.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch, HBASE-7482-v2.patch,
> HBASE-7482-v3.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601122#comment-13601122
]
Hudson commented on HBASE-7442:
---
Integrated in hbase-0.95-on-hadoop2 #24 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/24/])
HBASE-7482 Port HBASE-7442 HBase remote CopyTable not working when security
enabled to trunk (Revision 1455765)
Result = FAILURE
garyh :
Files :
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/ProtobufRpcClientEngine.java
*
/hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/RandomTimeoutRpcEngine.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestProtoBufRpc.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClusterId.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
>Assignee: James Kinley
> Fix For: 0.92.3, 0.94.4
>
> Attachments: attempt_201212271546_0001_m_00_0.log,
> HBASE-7442-0.92.1.patch, HBASE-7442-0.94.patch
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8066) Provide Admin.isTableAvailable() for a given table along with splitkeys
[ https://issues.apache.org/jira/browse/HBASE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8066: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.95. Thanks Chunhui for the review. > Provide Admin.isTableAvailable() for a given table along with splitkeys > --- > > Key: HBASE-8066 > URL: https://issues.apache.org/jira/browse/HBASE-8066 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8066_1.patch, HBASE-8066.patch > > > As part of HBASE-5583 if the master reboots during creation of table there is > a chance that the table gets created with partial split keys. This api helps > to check if the table was created with the required number of splitkeys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8063) Filter HFiles based on first/last key
[ https://issues.apache.org/jira/browse/HBASE-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601203#comment-13601203 ] Ted Yu commented on HBASE-8063: --- Integrated to trunk. Thanks for the patch, Liang. Thanks for the review, Chunhui. > Filter HFiles based on first/last key > - > > Key: HBASE-8063 > URL: https://issues.apache.org/jira/browse/HBASE-8063 > Project: HBase > Issue Type: Improvement > Components: Scanners >Affects Versions: 0.98.0 >Reporter: Liang Xie >Assignee: Liang Xie > Fix For: 0.98.0 > > Attachments: HBASE-8063.txt, HBASE-8063-v2.txt, HBASE-8063-v3.txt, > HBASE-8063-v4.txt > > > # hbase-5010 introduced a filter mechanism based on timerange hint > # this jira is a placeholder just for keyrange hint: currently, low > layer(hfile reader) has first/last key getter methods already. we should > utilize this information to pre-filte hfiles while choosing scanners from > storefiles. > It's just a raw thought right now, no patch available yet. I'll summit a > patch asap if it's reasonable, welcome to give some suggestions/comments:) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601206#comment-13601206 ] Ted Yu commented on HBASE-7824: --- Integrated to 0.94 Thanks for the patch, Jeff. Let's see how it goes. > Improve master start up time when there is log splitting work > - > > Key: HBASE-7824 > URL: https://issues.apache.org/jira/browse/HBASE-7824 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.94.7 > > Attachments: HBASE-7824_3.patch, hbase-7824.patch, > hbase-7824_v2.patch, hbase-7824_v3.patch > > > When there is log split work going on, master start up waits till all log > split work completes even though the log split has nothing to do with meta > region servers. > It's a bad behavior considering a master node can run when log split is > happening while its start up is blocking by log split work. > Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8012) Reseek should position to the beginning of file for the first time it is invoked with a KV smaller than the first KV in file
[ https://issues.apache.org/jira/browse/HBASE-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8012: -- Resolution: Fixed Fix Version/s: 0.95.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.95 and trunk. Thanks Lars and Stack for the review. Thanks Liu for working on this patch. > Reseek should position to the beginning of file for the first time it is > invoked with a KV smaller than the first KV in file > > > Key: HBASE-8012 > URL: https://issues.apache.org/jira/browse/HBASE-8012 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8012.patch, HBASE-8012.patch, HBASE-8012_v2.patch > > > The storeFileScanner's seekAtOrAfter method will position at the beginning of > the file when the passed KV is smaller than first KV in file. While for > reseekAtOrAfter, I think it should also do the same thing when it is the > first time it been seeked. originally, this is workaround by adding a > isReseekable property in StoreFileScanner, and is checked upon each > enforceSeek(), if it is not seeked before, it will go with seek approaching > instead of reseek approaching. While why not make reseekAtOrAfter working > correctly for the first time it been reseek (also never been seek before), > since the file is never seeked before, so position it at the beginning of the > file don't break the idea of "reseek", say never rewind. > It will save the effort for HBASE-8001, with this fixed, it won't need to > check isReseekable any more. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
stack created HBASE-8088: Summary: Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site Key: HBASE-8088 URL: https://issues.apache.org/jira/browse/HBASE-8088 Project: HBase Issue Type: Task Reporter: stack Doug Meil suggests its time we started versioning doc and site. First step is copying current version over to 0.94 so we have something to work against (site and book have only been kept up on trunk up to this). Let me do this much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-8088. -- Resolution: Fixed Fix Version/s: 0.94.7 Assignee: stack Committed copy of current trunk book and site to 0.94 branch. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601233#comment-13601233 ] Hudson commented on HBASE-8088: --- Integrated in HBase-0.94 #894 (See [https://builds.apache.org/job/HBase-0.94/894/]) HBASE-8088 Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site (Revision 1455996) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/docbkx/book.xml * /hbase/branches/0.94/src/docbkx/configuration.xml * /hbase/branches/0.94/src/docbkx/customization.xsl * /hbase/branches/0.94/src/docbkx/developer.xml * /hbase/branches/0.94/src/docbkx/external_apis.xml * /hbase/branches/0.94/src/docbkx/getting_started.xml * /hbase/branches/0.94/src/docbkx/ops_mgt.xml * /hbase/branches/0.94/src/docbkx/performance.xml * /hbase/branches/0.94/src/docbkx/preface.xml * /hbase/branches/0.94/src/docbkx/shell.xml * /hbase/branches/0.94/src/docbkx/troubleshooting.xml * /hbase/branches/0.94/src/docbkx/upgrading.xml * /hbase/branches/0.94/src/site/resources/css/site.css * /hbase/branches/0.94/src/site/resources/images/hbase_logo.png * /hbase/branches/0.94/src/site/resources/images/hbase_logo.svg * /hbase/branches/0.94/src/site/site.vm * /hbase/branches/0.94/src/site/site.xml * /hbase/branches/0.94/src/site/xdoc/acid-semantics.xml * /hbase/branches/0.94/src/site/xdoc/bulk-loads.xml * /hbase/branches/0.94/src/site/xdoc/cygwin.xml * /hbase/branches/0.94/src/site/xdoc/index.xml * /hbase/branches/0.94/src/site/xdoc/metrics.xml * /hbase/branches/0.94/src/site/xdoc/old_news.xml * /hbase/branches/0.94/src/site/xdoc/pseudo-distributed.xml * /hbase/branches/0.94/src/site/xdoc/replication.xml * /hbase/branches/0.94/src/site/xdoc/sponsors.xml > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601234#comment-13601234 ] Hudson commented on HBASE-7824: --- Integrated in HBase-0.94 #894 (See [https://builds.apache.org/job/HBase-0.94/894/]) HBASE-7824 Improve master start up time when there is log splitting work (Jeffrey Zhong) (Revision 1455976) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenMasterInitializing.java > Improve master start up time when there is log splitting work > - > > Key: HBASE-7824 > URL: https://issues.apache.org/jira/browse/HBASE-7824 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.94.7 > > Attachments: HBASE-7824_3.patch, hbase-7824.patch, > hbase-7824_v2.patch, hbase-7824_v3.patch > > > When there is log split work going on, master start up waits till all log > split work completes even though the log split has nothing to do with meta > region servers. > It's a bad behavior considering a master node can run when log split is > happening while its start up is blocking by log split work. > Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601236#comment-13601236 ] Andrew Purtell commented on HBASE-7992: --- bq. What do you say Andrew? May be we can handle that in another JIRA as that will be of wider scope. No, this instance should be dealt with in the context of this review. I am -1 only because these are new hooks with an issue called out in review. Don't push it off, address it here before the new changes go in. > provide pre/post region offline hooks for HMaster.offlineRegion(). > --- > > Key: HBASE-7992 > URL: https://issues.apache.org/jira/browse/HBASE-7992 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Affects Versions: 0.95.0 >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.0 > > Attachments: HBASE-7992_trunk.patch > > > presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601240#comment-13601240 ] Andrew Purtell commented on HBASE-7992: --- I suggest one of the following approaches: 1. Update the javadoc to indicate that bypass() will not work here. Remove the logic that does the bypass at the hook site to insure bypass is not possible. 2. Modify the hook and the OfflineRegionResponse response so we can communicate back to the caller that the offlining was ignored. Probably option 1 is easiest. Either one of these is fine, I'll remove my veto. > provide pre/post region offline hooks for HMaster.offlineRegion(). > --- > > Key: HBASE-7992 > URL: https://issues.apache.org/jira/browse/HBASE-7992 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Affects Versions: 0.95.0 >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.0 > > Attachments: HBASE-7992_trunk.patch > > > presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8087) Add result information in ProtoBuf response object for master/rs ops
[ https://issues.apache.org/jira/browse/HBASE-8087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-8087. --- Resolution: Invalid Please resubmit with a clear description of the issue and what is the objective. > Add result information in ProtoBuf response object for master/rs ops > > > Key: HBASE-8087 > URL: https://issues.apache.org/jira/browse/HBASE-8087 > Project: HBase > Issue Type: Improvement >Reporter: rajeshbabu >Assignee: rajeshbabu >Priority: Minor > > Pls refer to the comments > https://issues.apache.org/jira/browse/HBASE-7992?focusedCommentId=13600513&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13600513 > https://issues.apache.org/jira/browse/HBASE-7992?focusedCommentId=13600780&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13600780 > Raised this issue to solve that comments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[
https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601247#comment-13601247
]
Lars Hofhansl commented on HBASE-8088:
--
Hmm... 0.94 fails with:
{code}
Recoverable error
org.xml.sax.SAXParseException: Include operation failed, reverting to fallback.
Resource error reading file as XML
(href='../../hbase-common/src/main/resources/hbase-default.xml'). Reason:
/home/jenkins/jenkins-slave/workspace/HBase-0.94/trunk/hbase-common/src/main/resources/hbase-default.xml
(No such file or directory)
Error on line 672 column 74 of
file:///home/jenkins/jenkins-slave/workspace/HBase-0.94/trunk/src/docbkx/configuration.xml:
Error reported by XML parser: An 'include' failed, and no 'fallback' element
was found.
{code}
(Just copied it here from jenkins so we have it here)
> Versioning site: part one, put stake in the ground for 0.94 by copying
> current versions of book and site
>
>
> Key: HBASE-8088
> URL: https://issues.apache.org/jira/browse/HBASE-8088
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.7
>
>
> Doug Meil suggests its time we started versioning doc and site. First step
> is copying current version over to 0.94 so we have something to work against
> (site and book have only been kept up on trunk up to this). Let me do this
> much for now. Can figure how to do the rest later.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8028) Append, Increment: Adding rollback support
[ https://issues.apache.org/jira/browse/HBASE-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601290#comment-13601290 ] Himanshu Vashishtha commented on HBASE-8028: Please let me know if you have any other comments/suggestions. > Append, Increment: Adding rollback support > -- > > Key: HBASE-8028 > URL: https://issues.apache.org/jira/browse/HBASE-8028 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.0 > > Attachments: HBase-8028-v1.patch, HBase-8028-v2.patch, > HBase-8028-with-Increments-v1.patch > > > In case there is an exception while doing the log-sync, the memstore is not > rollbacked, while the mvcc is _always_ forwarded to the writeentry created at > the beginning of the operation. This may lead to scanners seeing results > which are not synched to the fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8089) Add type support
Nick Dimiduk created HBASE-8089: --- Summary: Add type support Key: HBASE-8089 URL: https://issues.apache.org/jira/browse/HBASE-8089 Project: HBase Issue Type: New Feature Components: Client Reporter: Nick Dimiduk Attachments: HBASE-8089-types.txt This proposal outlines an improvement to HBase that provides for a set of types, above and beyond the existing "byte-bucket" strategy. This is intended to reduce user-level duplication of effort, provide better support for 3rd-party integration, and provide an overall improved experience for developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601298#comment-13601298
]
Tianying Chang commented on HBASE-8085:
---
Sure, I will put the patch.
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8089) Add type support
[ https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8089: Attachment: HBASE-8089-types.txt > Add type support > > > Key: HBASE-8089 > URL: https://issues.apache.org/jira/browse/HBASE-8089 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Nick Dimiduk > Attachments: HBASE-8089-types.txt > > > This proposal outlines an improvement to HBase that provides for a set of > types, above and beyond the existing "byte-bucket" strategy. This is intended > to reduce user-level duplication of effort, provide better support for > 3rd-party integration, and provide an overall improved experience for > developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Issue Type: Sub-task (was: Improvement) Parent: HBASE-8089 > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Sub-task > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7221) RowKey utility class for rowkey construction
[
https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-7221:
Issue Type: Sub-task (was: Improvement)
Parent: HBASE-8089
> RowKey utility class for rowkey construction
>
>
> Key: HBASE-7221
> URL: https://issues.apache.org/jira/browse/HBASE-7221
> Project: HBase
> Issue Type: Sub-task
> Components: util
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch,
> hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch,
> hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch
>
>
> A common question in the dist-lists is how to construct rowkeys, particularly
> composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to
> you to sensibly populate that byte-array, and that's where things tend to go
> off the rails.
> The intent of this RowKey utility class isn't meant to add functionality into
> Put/Get/Scan, but rather make it simpler for folks to construct said arrays.
> Example:
> {code}
>RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG);
>key.addHash(a);
>key.add(b);
>byte bytes[] = key.getBytes();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7941) Provide client API with support for primitive types
[ https://issues.apache.org/jira/browse/HBASE-7941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7941: Issue Type: Sub-task (was: Improvement) Parent: HBASE-8089 > Provide client API with support for primitive types > --- > > Key: HBASE-7941 > URL: https://issues.apache.org/jira/browse/HBASE-7941 > Project: HBase > Issue Type: Sub-task > Components: Client, Usability >Reporter: Nick Dimiduk > > Work is underway to provide a widely acceptable serialization format for > primitive and complex types (HBASE-7221, HBASE-7692). With this completed, > those serialization conveniences should be pushed up to users of the Client > API by way of additional method signatures on Operation implementations (Get, > Put, Delete, Scan, &c.). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Status: Open (was: Patch Available) > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Sub-task > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-7692. - Resolution: Won't Fix Fix Version/s: (was: 0.95.0) Dropping in Orderly is not the right approach. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Sub-task > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) [experiment] Adopt Orderly library
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Summary: [experiment] Adopt Orderly library (was: Add utility class to generate ordered byte[] serialization) > [experiment] Adopt Orderly library > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Sub-task > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7221) [experiment] RowKey utility class for rowkey construction
[
https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-7221:
Summary: [experiment] RowKey utility class for rowkey construction (was:
RowKey utility class for rowkey construction)
> [experiment] RowKey utility class for rowkey construction
> -
>
> Key: HBASE-7221
> URL: https://issues.apache.org/jira/browse/HBASE-7221
> Project: HBase
> Issue Type: Sub-task
> Components: util
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch,
> hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch,
> hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch
>
>
> A common question in the dist-lists is how to construct rowkeys, particularly
> composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to
> you to sensibly populate that byte-array, and that's where things tend to go
> off the rails.
> The intent of this RowKey utility class isn't meant to add functionality into
> Put/Get/Scan, but rather make it simpler for folks to construct said arrays.
> Example:
> {code}
>RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG);
>key.addHash(a);
>key.add(b);
>byte bytes[] = key.getBytes();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Upgrade hadoop 2.0 dependency to 2.0.4-alpha
[
https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601329#comment-13601329
]
Ted Yu commented on HBASE-7904:
---
>From
>https://builds.apache.org/job/PreCommit-HBASE-Build/4788//testReport/org.apache.hadoop.hbase.mapreduce/TestImportExport/testExportScannerBatching/:
{code}
2013-03-12 23:54:06,948 INFO [IPC Server handler 7 on 42466]
blockmanagement.BlockManager(969): BLOCK* addToInvalidates:
blk_-7803627020849108776_1036 127.0.0.1:46824
2013-03-12 23:54:07,856 WARN [IPC Server handler 3 on 47555]
containermanager.ContainerManagerImpl(497): Trying to stop unknown container
container_1363132417629_0001_01_01
2013-03-12 23:54:07,858 WARN [IPC Server handler 3 on 47555]
nodemanager.NMAuditLogger(150): USER=UnknownUser IP=67.195.138.30
OPERATION=Stop Container RequestTARGET=ContainerManagerImpl
RESULT=FAILURE DESCRIPTION=Trying to stop unknown container!
APPID=application_1363132417629_0001
CONTAINERID=container_1363132417629_0001_01_01
...
2013-03-12 23:54:24,952 INFO [IPC Server handler 1 on 42466]
blockmanagement.BlockManager(2174): BLOCK* addStoredBlock: blockMap updated:
127.0.0.1:46824 is added to
blk_383661822379709623_1109{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:46824|RBW]]}
size 0
2013-03-12 23:54:25,129 WARN [ContainersLauncher #0]
nodemanager.DefaultContainerExecutor(195): Exit code from task is : 137
2013-03-12 23:54:29,492 INFO [IPC Server handler 0 on 42466]
blockmanagement.BlockManager(969): BLOCK* addToInvalidates:
blk_7203999672823138585_1053 127.0.0.1:46824
2013-03-12 23:54:29,516 INFO [pool-1-thread-1] hbase.ResourceChecker(171):
after: mapreduce.TestImportExport#testSimpleCase Thread=509 (was 468)
{code}
TestImportExport#testSimpleCase didn't fail. So exit code 137 may not be fatal.
> Upgrade hadoop 2.0 dependency to 2.0.4-alpha
>
>
> Key: HBASE-7904
> URL: https://issues.apache.org/jira/browse/HBASE-7904
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: 7904.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt,
> 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt,
> hbase-7904-v3.txt
>
>
> 2.0.3-alpha has been released.
> We should upgrade the dependency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601333#comment-13601333 ] Gary Helmling commented on HBASE-8088: -- Yes! We need this. We've had issues come up in the security docs, where there is confusion over how to config security in 0.94 because some parameters have been removed from the documentation in trunk (which is the one that is published). > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8088: - Attachment: addendum.txt Sorry lars. Here is fix. Committed. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8090) Versioning site; part two, publish 0.94 site and add link from main site
stack created HBASE-8090: Summary: Versioning site; part two, publish 0.94 site and add link from main site Key: HBASE-8090 URL: https://issues.apache.org/jira/browse/HBASE-8090 Project: HBase Issue Type: Task Reporter: stack Priority: Blocker Do the rest of site versioning. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601353#comment-13601353 ] stack commented on HBASE-8088: -- Added HBASE-8090 for part2. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8034) record on-disk data size for store file and make it available during writing
[
https://issues.apache.org/jira/browse/HBASE-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601361#comment-13601361
]
Nick Dimiduk commented on HBASE-8034:
-
{noformat}
--- hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
+++ hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
@@ -319,6 +319,11 @@ public class HFile {
* HFile V2.
*/
void addDeleteFamilyBloomFilter(BloomFilterWriter bfw) throws IOException;
+
+/**
+ * @return Currently written raw data size on disk.
+ */
+long getCurrentSize() throws IOException;
}
{noformat}
nit: "\@return Currently written raw data size on disk *or -1.*"
Otherwise, ship it!
> record on-disk data size for store file and make it available during writing
>
>
> Key: HBASE-8034
> URL: https://issues.apache.org/jira/browse/HBASE-8034
> Project: HBase
> Issue Type: Task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Minor
> Attachments: HBASE-8034-v0.patch, HBASE-8034-v1.patch,
> HBASE-8034-v2.patch
>
>
> To better estimate the size of data in the file, and to be able to split
> files intelligently during any multi-file compactor like stripe or level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7697) Consolidate tools for getting data into, out of HBase
[ https://issues.apache.org/jira/browse/HBASE-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-7697. - Resolution: Invalid Closing as invalid because this is pretty vague. If you're interested, see related mapreduce improvements in HBASE-8084. > Consolidate tools for getting data into, out of HBase > - > > Key: HBASE-7697 > URL: https://issues.apache.org/jira/browse/HBASE-7697 > Project: HBase > Issue Type: Improvement > Components: Client, mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > > The user experience for importing data into HBase and getting a dump out of > HBase is pretty poor. The existing tools as I understand them include: > - org.apache.hadoop.hbase.mapreduce.Export, > - org.apache.hadoop.hbase.mapreduce.Import, > - org.apache.hadoop.hbase.mapreduce.ImportTsv, > - org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles, and > - org.apache.hadoop.hbase.mapreduce.CopyTable > Each one provides specific features that do not necessarily overlap with the > others. For instance, Import and ImportTsv could have most of their logic > combined, sharing common driver code and leaving the details of the > file-format up to the user to provide via a pluggable mapper. Export and > CopyTable both map over a target table; it's only the detail of what they do > with the data that is different. Bulk operations via HFiles could be a more > common use-case as well, not just a special case of ImportTsv. > The list of [open > issues|https://issues.apache.org/jira/issues/?filter=-1&jql=project%20%3D%20HBASE%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20text%20~%20%22ImportTsv%22%20ORDER%20BY%20updatedDate%20DESC] > against ImportTsv alone indicates users are using the tool, and I certainly > advise it for people getting started with a new HBase deployment. > I propose a single interface for getting data into and out of HBase. It would > be pluggable, allowing users to override details of their file formats and > schemas. We can provide implementations that replicate existing tool > behaviors as example modules. These tools are also a reasonable place, IMHO, > to include support for creation and loading of snapshots. > I started down the path of a specific tool intended to overcome some of the > limitations of ImportTsv and it has since refactored into a more general > purpose application. Initial patches forthcoming. Comments strongly > encouraged. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8035) Add site target check to precommit tests
[ https://issues.apache.org/jira/browse/HBASE-8035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601365#comment-13601365 ] Ted Yu commented on HBASE-8035: --- So, which addendum should we use ? > Add site target check to precommit tests > > > Key: HBASE-8035 > URL: https://issues.apache.org/jira/browse/HBASE-8035 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: Nick Dimiduk > Fix For: 0.98.0 > > Attachments: > 0001-HBASE-8035-Add-site-generation-to-patch-validation.patch, > 8035-addendum.txt, 8035-addendum.txt, addendum.out > > > We should check that the Maven 'site' target passes as part of precommit > testing. See HBASE-8022. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8081) Backport HBASE-7213 (separate hlog for meta tables) to 0.94
[
https://issues.apache.org/jira/browse/HBASE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601377#comment-13601377
]
Ted Yu commented on HBASE-8081:
---
Looks like patch v2 doesn't apply cleanly to 0.94:
{code}
Hunk #6 succeeded at 349 (offset 1 line).
1 out of 6 hunks FAILED -- saving rejects to file
src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java.rej
{code}
> Backport HBASE-7213 (separate hlog for meta tables) to 0.94
> ---
>
> Key: HBASE-8081
> URL: https://issues.apache.org/jira/browse/HBASE-8081
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.5
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Fix For: 0.94.7
>
> Attachments: 7213-0.94-2.patch, 7213-0.94.patch
>
>
> I am interested in backporting HBASE-7213 to 0.94. Helps to address more of
> the MTTR story. Offline discussion with Lars indicated he is interested as
> well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianying Chang updated HBASE-8085:
--
Attachment: HBASE-8085.patch
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8036) ProtobufUtil.multi behavior is inconsistent in case of errors
[ https://issues.apache.org/jira/browse/HBASE-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601392#comment-13601392 ] Sergey Shelukhin commented on HBASE-8036: - +1 > ProtobufUtil.multi behavior is inconsistent in case of errors > - > > Key: HBASE-8036 > URL: https://issues.apache.org/jira/browse/HBASE-8036 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.95.0 > > Attachments: hbase-8036_v1.patch, hbase-8036_v2.patch, > hbase-8036_v3.patch, hbase-8036_v4.patch > > > ProtobufUtil splits operations by regions and performs multiple client.multi > calls. In case if there are certain errors inside RS, HRegionServer adds the > corresponding exceptions to MultiResponse, PU continues the multi request for > other regions, and returns partial failure. > In case of other errors (for example, region not served exception), the > entire multi operation stops executing, and previous successes and partial > results are disregarded. > ProtobufUtil should probably catch ServiceException separately for each > client.multi call, make it a partial-failure exception for all actions for > this region, and also continue the batch, to make the behavior consistent. > Alternatively, if we want to avoid continuing the batch in case of some > server-wide errors/connection problems/etc., server should do that for > region-specific errors (add exception to results for each action). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianying Chang updated HBASE-8085:
--
Status: Patch Available (was: Open)
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7948) client doesn't need to refresh meta while the region is opening
[ https://issues.apache.org/jira/browse/HBASE-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601400#comment-13601400 ] Sergey Shelukhin commented on HBASE-7948: - bq. HRegionCacheLocation just has seqid in it now. Is that enough to justify separate class? If you left it in RegionLocation, your patch would be a good bit smaller again (smile). I think it is better decision for the future, if we want to add more information. Look at all the test code that got cleaned up because it never cared about seqid :) bq. Is the class comment on RegionOpeningException right? Is it copy/paste? The locating-it code, yes. > client doesn't need to refresh meta while the region is opening > --- > > Key: HBASE-7948 > URL: https://issues.apache.org/jira/browse/HBASE-7948 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7948-v0.patch, HBASE-7948-v1.patch, > HBASE-7948-v1.patch, HBASE-7948-v2.patch, HBASE-7948-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601402#comment-13601402
]
Hadoop QA commented on HBASE-8085:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573536/HBASE-8085.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4797//console
This message is automatically generated.
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8081) Backport HBASE-7213 (separate hlog for meta tables) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601403#comment-13601403 ] Devaraj Das commented on HBASE-8081: Yeah, it seems HBASE-7824 commit has led to some conflicts with this patch. Looking.. > Backport HBASE-7213 (separate hlog for meta tables) to 0.94 > --- > > Key: HBASE-8081 > URL: https://issues.apache.org/jira/browse/HBASE-8081 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.5 >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.94.7 > > Attachments: 7213-0.94-2.patch, 7213-0.94.patch > > > I am interested in backporting HBASE-7213 to 0.94. Helps to address more of > the MTTR story. Offline discussion with Lars indicated he is interested as > well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7679) implement store file management for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601406#comment-13601406 ] Sergey Shelukhin commented on HBASE-7679: - Ping? [~mbertozzi] [~stack] what do you guys think about how this should go in? All patches (HBASE-7679 HBASE-7680 HBASE-7967 HBASE-8000) together, or separately? My plan was to commit separately to trunk only for now > implement store file management for stripe compactions > -- > > Key: HBASE-7679 > URL: https://issues.apache.org/jira/browse/HBASE-7679 > Project: HBase > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7667-and-7603-v0-incomplete.patch, > HBASE-7667-and-7603-v0-incomplete.patch, HBASE-7667-and-7603-v1.patch, > HBASE-7667-and-7603-v1.patch, HBASE-7667-v1.patch, HBASE-7667-v1.patch, > HBASE-7667-v2.patch, HBASE-7667-v2.patch, HBASE-7667-v3.patch, > HBASE-7679-v4.patch, HBASE-7679-v5.patch, HBASE-7679-v6.patch, > HBASE-7679-v7-.patch, HBASE-7679-v7.patch, HBASE-7679-v8.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7791) Compaction USER_PRIORITY is slightly broken
[
https://issues.apache.org/jira/browse/HBASE-7791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601409#comment-13601409
]
Sergey Shelukhin commented on HBASE-7791:
-
Will commit soon-ish if there are no objections.
> Compaction USER_PRIORITY is slightly broken
> ---
>
> Key: HBASE-7791
> URL: https://issues.apache.org/jira/browse/HBASE-7791
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Minor
> Attachments: HBASE-7791-v0.patch, HBASE-7791-v1.patch
>
>
> The code to get compaction priority is as such:
> {code} public int getCompactPriority(int priority) {
> // If this is a user-requested compaction, leave this at the highest
> priority
> if(priority == Store.PRIORITY_USER) {
>return Store.PRIORITY_USER;
> } else {
>return this.blockingStoreFileCount - this.storefiles.size();
> }
>}
> {code}.
> PRIORITY_USER is 1.
> The priorities are compared as numbers in HRegion, so compactions of blocking
> stores will override user priority (probably intended); also, if you have
> blockingFiles minus one, your priority is suddenly PRIORITY_USER, which may
> cause at least this:
> LOG.debug("Warning, compacting more than " +
> comConf.getMaxFilesToCompact() +
> " files because of a user-requested major compaction");
> as well as some misleading logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8012) Reseek should position to the beginning of file for the first time it is invoked with a KV smaller than the first KV in file
[ https://issues.apache.org/jira/browse/HBASE-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601415#comment-13601415 ] Hudson commented on HBASE-8012: --- Integrated in hbase-0.95 #69 (See [https://builds.apache.org/job/hbase-0.95/69/]) HBASE-8012 - Reseek should position to the beginning of file for the first time it is invoked with a KV smaller than the first KV in file (Raymond Liu) (Revision 1455990) Result = FAILURE ramkrishna : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java > Reseek should position to the beginning of file for the first time it is > invoked with a KV smaller than the first KV in file > > > Key: HBASE-8012 > URL: https://issues.apache.org/jira/browse/HBASE-8012 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8012.patch, HBASE-8012.patch, HBASE-8012_v2.patch > > > The storeFileScanner's seekAtOrAfter method will position at the beginning of > the file when the passed KV is smaller than first KV in file. While for > reseekAtOrAfter, I think it should also do the same thing when it is the > first time it been seeked. originally, this is workaround by adding a > isReseekable property in StoreFileScanner, and is checked upon each > enforceSeek(), if it is not seeked before, it will go with seek approaching > instead of reseek approaching. While why not make reseekAtOrAfter working > correctly for the first time it been reseek (also never been seek before), > since the file is never seeked before, so position it at the beginning of the > file don't break the idea of "reseek", say never rewind. > It will save the effort for HBASE-8001, with this fixed, it won't need to > check isReseekable any more. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8066) Provide Admin.isTableAvailable() for a given table along with splitkeys
[ https://issues.apache.org/jira/browse/HBASE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601416#comment-13601416 ] Hudson commented on HBASE-8066: --- Integrated in hbase-0.95 #69 (See [https://builds.apache.org/job/hbase-0.95/69/]) HBASE-8066 - Provide Admin.isTableAvailable() for a given table along with splitkeys (Ram) (Revision 1455962) Result = FAILURE ramkrishna : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java > Provide Admin.isTableAvailable() for a given table along with splitkeys > --- > > Key: HBASE-8066 > URL: https://issues.apache.org/jira/browse/HBASE-8066 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8066_1.patch, HBASE-8066.patch > > > As part of HBASE-5583 if the master reboots during creation of table there is > a chance that the table gets created with partial split keys. This api helps > to check if the table was created with the required number of splitkeys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8066) Provide Admin.isTableAvailable() for a given table along with splitkeys
[ https://issues.apache.org/jira/browse/HBASE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601418#comment-13601418 ] Hudson commented on HBASE-8066: --- Integrated in HBase-TRUNK #3954 (See [https://builds.apache.org/job/HBase-TRUNK/3954/]) HBASE-8066 - Provide Admin.isTableAvailable() for a given table along with splitkeys(Ram) (Revision 1455958) Result = FAILURE ramkrishna : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java > Provide Admin.isTableAvailable() for a given table along with splitkeys > --- > > Key: HBASE-8066 > URL: https://issues.apache.org/jira/browse/HBASE-8066 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8066_1.patch, HBASE-8066.patch > > > As part of HBASE-5583 if the master reboots during creation of table there is > a chance that the table gets created with partial split keys. This api helps > to check if the table was created with the required number of splitkeys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8012) Reseek should position to the beginning of file for the first time it is invoked with a KV smaller than the first KV in file
[ https://issues.apache.org/jira/browse/HBASE-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601417#comment-13601417 ] Hudson commented on HBASE-8012: --- Integrated in HBase-TRUNK #3954 (See [https://builds.apache.org/job/HBase-TRUNK/3954/]) HBASE-8012 - Reseek should position to the beginning of file for the first time it is invoked with a KV smaller than the first KV in file (Raymond Liu) (Revision 1455993) Result = FAILURE ramkrishna : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java > Reseek should position to the beginning of file for the first time it is > invoked with a KV smaller than the first KV in file > > > Key: HBASE-8012 > URL: https://issues.apache.org/jira/browse/HBASE-8012 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8012.patch, HBASE-8012.patch, HBASE-8012_v2.patch > > > The storeFileScanner's seekAtOrAfter method will position at the beginning of > the file when the passed KV is smaller than first KV in file. While for > reseekAtOrAfter, I think it should also do the same thing when it is the > first time it been seeked. originally, this is workaround by adding a > isReseekable property in StoreFileScanner, and is checked upon each > enforceSeek(), if it is not seeked before, it will go with seek approaching > instead of reseek approaching. While why not make reseekAtOrAfter working > correctly for the first time it been reseek (also never been seek before), > since the file is never seeked before, so position it at the beginning of the > file don't break the idea of "reseek", say never rewind. > It will save the effort for HBASE-8001, with this fixed, it won't need to > check isReseekable any more. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8063) Filter HFiles based on first/last key
[ https://issues.apache.org/jira/browse/HBASE-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601419#comment-13601419 ] Hudson commented on HBASE-8063: --- Integrated in HBase-TRUNK #3954 (See [https://builds.apache.org/job/HBase-TRUNK/3954/]) HBASE-8063 Filter HFiles based on first/last key (Liang Xie) (Revision 1455971) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingKeyRange.java > Filter HFiles based on first/last key > - > > Key: HBASE-8063 > URL: https://issues.apache.org/jira/browse/HBASE-8063 > Project: HBase > Issue Type: Improvement > Components: Scanners >Affects Versions: 0.98.0 >Reporter: Liang Xie >Assignee: Liang Xie > Fix For: 0.98.0 > > Attachments: HBASE-8063.txt, HBASE-8063-v2.txt, HBASE-8063-v3.txt, > HBASE-8063-v4.txt > > > # hbase-5010 introduced a filter mechanism based on timerange hint > # this jira is a placeholder just for keyrange hint: currently, low > layer(hfile reader) has first/last key getter methods already. we should > utilize this information to pre-filte hfiles while choosing scanners from > storefiles. > It's just a raw thought right now, no patch available yet. I'll summit a > patch asap if it's reasonable, welcome to give some suggestions/comments:) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7905) Add passing of optional cell blocks over rpc
[ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7905: - Attachment: 7905v8.txt Rebase > Add passing of optional cell blocks over rpc > > > Key: HBASE-7905 > URL: https://issues.apache.org/jira/browse/HBASE-7905 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack > Fix For: 0.95.0 > > Attachments: 7905.txt, 7905v3.txt, 7905v4.txt, 7905v6.txt, 7905v8.txt > > > Make it so we can pass Cells/data w/o having to bury it all in protobuf to > get it over the wire. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7905) Add passing of optional cell blocks over rpc
[
https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601427#comment-13601427
]
Hadoop QA commented on HBASE-7905:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573539/7905v8.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 48 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4798//console
This message is automatically generated.
> Add passing of optional cell blocks over rpc
>
>
> Key: HBASE-7905
> URL: https://issues.apache.org/jira/browse/HBASE-7905
> Project: HBase
> Issue Type: Sub-task
> Components: IPC/RPC
>Reporter: stack
> Fix For: 0.95.0
>
> Attachments: 7905.txt, 7905v3.txt, 7905v4.txt, 7905v6.txt, 7905v8.txt
>
>
> Make it so we can pass Cells/data w/o having to bury it all in protobuf to
> get it over the wire.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8036) ProtobufUtil.multi behavior is inconsistent in case of errors
[ https://issues.apache.org/jira/browse/HBASE-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8036: - Resolution: Fixed Fix Version/s: 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've committed this. Thanks for reviews. > ProtobufUtil.multi behavior is inconsistent in case of errors > - > > Key: HBASE-8036 > URL: https://issues.apache.org/jira/browse/HBASE-8036 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-8036_v1.patch, hbase-8036_v2.patch, > hbase-8036_v3.patch, hbase-8036_v4.patch > > > ProtobufUtil splits operations by regions and performs multiple client.multi > calls. In case if there are certain errors inside RS, HRegionServer adds the > corresponding exceptions to MultiResponse, PU continues the multi request for > other regions, and returns partial failure. > In case of other errors (for example, region not served exception), the > entire multi operation stops executing, and previous successes and partial > results are disregarded. > ProtobufUtil should probably catch ServiceException separately for each > client.multi call, make it a partial-failure exception for all actions for > this region, and also continue the batch, to make the behavior consistent. > Alternatively, if we want to avoid continuing the batch in case of some > server-wide errors/connection problems/etc., server should do that for > region-specific errors (add exception to results for each action). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8091) Goraci test might rewrite data in case of map task failures, which eclipses data loss problems
Enis Soztutar created HBASE-8091: Summary: Goraci test might rewrite data in case of map task failures, which eclipses data loss problems Key: HBASE-8091 URL: https://issues.apache.org/jira/browse/HBASE-8091 Project: HBase Issue Type: Bug Components: test Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.95.0, 0.98.0 As discussed in HBASE-8031, goraci map tasks will rewite the same data if the map task fails. There are some conditions that this might cause overwriting data that would have been reported as loss otherwise. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8081) Backport HBASE-7213 (separate hlog for meta tables) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-8081: --- Attachment: 7213-0.94-3.patch This should address the problem. Testing this patch presently. > Backport HBASE-7213 (separate hlog for meta tables) to 0.94 > --- > > Key: HBASE-8081 > URL: https://issues.apache.org/jira/browse/HBASE-8081 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.5 >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.94.7 > > Attachments: 7213-0.94-2.patch, 7213-0.94-3.patch, 7213-0.94.patch > > > I am interested in backporting HBASE-7213 to 0.94. Helps to address more of > the MTTR story. Offline discussion with Lars indicated he is interested as > well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7590) Add a costless notifications mechanism from master to regionservers & clients
[ https://issues.apache.org/jira/browse/HBASE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601449#comment-13601449 ] nkeywal commented on HBASE-7590: I will fix the 100 lines stuff on commit. Any +1 on the new version? > Add a costless notifications mechanism from master to regionservers & clients > - > > Key: HBASE-7590 > URL: https://issues.apache.org/jira/browse/HBASE-7590 > Project: HBase > Issue Type: Bug > Components: Client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Attachments: 7590.inprogress.patch, 7590.v1.patch, > 7590.v1-rebased.patch, 7590.v2.patch, 7590.v3.patch, 7590.v5.patch, > 7590.v5.patch > > > t would be very useful to add a mechanism to distribute some information to > the clients and regionservers. Especially It would be useful to know globally > (regionservers + clients apps) that some regionservers are dead. This would > allow: > - to lower the load on the system, without clients using staled information > and going on dead machines > - to make the recovery faster from a client point of view. It's common to use > large timeouts on the client side, so the client may need a lot of time > before declaring a region server dead and trying another one. If the client > receives the information separatly about a region server states, it can take > the right decision, and continue/stop to wait accordingly. > We can also send more information, for example instructions like 'slow down' > to instruct the client to increase the retries delay and so on. > Technically, the master could send this information. To lower the load on > the system, we should: > - have a multicast communication (i.e. the master does not have to connect to > all servers by tcp), with once packet every 10 seconds or so. > - receivers should not depend on this: if the information is available great. > If not, it should not break anything. > - it should be optional. > So at the end we would have a thread in the master sending a protobuf message > about the dead servers on a multicast socket. If the socket is not > configured, it does not do anything. On the client side, when we receive an > information that a node is dead, we refresh the cache about it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7840) Enhance the java it framework to start & stop a distributed hbase & hadoop cluster
[ https://issues.apache.org/jira/browse/HBASE-7840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601454#comment-13601454 ] nkeywal commented on HBASE-7840: Waiting for review or +1 on this one, I can rebase if you like. > Enhance the java it framework to start & stop a distributed hbase & hadoop > cluster > --- > > Key: HBASE-7840 > URL: https://issues.apache.org/jira/browse/HBASE-7840 > Project: HBase > Issue Type: New Feature > Components: test >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.95.0 > > Attachments: 7840.v1.patch, 7840.v3.patch > > > Needs are to use a development version of HBase & HDFS 1 & 2. > Ideally, should be nicely backportable to 0.94 to allow comparisons and > regression tests between versions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601467#comment-13601467
]
Enis Soztutar commented on HBASE-8085:
--
Hadoopqa reports that it cannot apply the patch. Can you please rebase?
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601469#comment-13601469
]
Ted Yu commented on HBASE-8085:
---
The patch applies cleanly.
Hadoop QA only applies patch on trunk.
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8077) Configure the job name in ImportTsv
[ https://issues.apache.org/jira/browse/HBASE-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601473#comment-13601473 ] udy brill commented on HBASE-8077: -- with pleasure! > Configure the job name in ImportTsv > --- > > Key: HBASE-8077 > URL: https://issues.apache.org/jira/browse/HBASE-8077 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: udy brill >Priority: Minor > Labels: importtsv, mapreduce > Original Estimate: 48h > Remaining Estimate: 48h > > Currently when using the ImportTsv tool, the name of the mapreduce job is > hard coded and cannot be change (importtsv_tableName) > it would be nice if this was the default, but could be change. > an idea is: > bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv > -Dimporttsv.columns=a,b,c -Dmapred.job.name=newJobName > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianying Chang updated HBASE-8085:
--
Attachment: HBASE-8085-v2.patch
rebased and new patch attached. Thanks.
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601488#comment-13601488
]
Hadoop QA commented on HBASE-8085:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573552/HBASE-8085-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4799//console
This message is automatically generated.
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[
https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601490#comment-13601490
]
Lars Hofhansl commented on HBASE-8088:
--
Now it's doing this:
{code}
Recoverable error
org.xml.sax.SAXParseException: Include operation failed, reverting to fallback.
Resource error reading file as XML (href='security.xml'). Reason:
/home/jenkins/jenkins-slave/workspace/HBase-0.94/trunk/src/docbkx/security.xml
(No such file or directory)
Error on line 1343 column 80 of
file:///home/jenkins/jenkins-slave/workspace/HBase-0.94/trunk/src/docbkx/book.xml:
Error reported by XML parser: An 'include' failed, and no 'fallback' element
was found.
{code}
I can't find security.xml anywhere.
> Versioning site: part one, put stake in the ground for 0.94 by copying
> current versions of book and site
>
>
> Key: HBASE-8088
> URL: https://issues.apache.org/jira/browse/HBASE-8088
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.7
>
> Attachments: addendum.txt
>
>
> Doug Meil suggests its time we started versioning doc and site. First step
> is copying current version over to 0.94 so we have something to work against
> (site and book have only been kept up on trunk up to this). Let me do this
> much for now. Can figure how to do the rest later.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601513#comment-13601513 ] Nick Dimiduk commented on HBASE-7938: - bq. change the test method name generateAndLoad() to start with "test" per convention Can do. bq. One difference between running from command line and running by: HBASE_CLASSPATH=hbase-it/target/hbase-it-0.97-SNAPSHOT-tests.jar bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver -test IntegrationTestImportTsv is that the latter will run all of the tests (both TestImportTsv and IntegrationTestImportTsv) while the former will only run the generateAndLoad() test That's true. I considered using a reflection loop to execute all of the tests under this scenario, but I decided it wasn't really worth it. Maybe it could be done by programmatically driving JUnit, but getting the conf objects correct looks surprisingly difficult. bq. Also got this exception while running from IntegrationTestsDriver This looks like the IntegrationTestingUtility didn't clean up the test directory after a run. Did you kill or have an error on a previous run? > Add integration test for ImportTsv/LoadIncrementalHFiles workflow > - > > Key: HBASE-7938 > URL: https://issues.apache.org/jira/browse/HBASE-7938 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0, 0.98.0 > > Attachments: > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch > > > We have existing unit tests for smoke-testing the packaged MR jobs, however > they do not create a runtime environment that is true to running on a real MR > cluster. This is particularly true in regard to classpaths (HBASE-7934) but > also other static state (HBASE-4802). An integration test that can be pointed > to run on a pseudo-distributed Hadoop deployed on localhost would find these > kinds of problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601528#comment-13601528 ] Lars Hofhansl commented on HBASE-8088: -- That seems to exist in trunk only. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601528#comment-13601528 ] Lars Hofhansl edited comment on HBASE-8088 at 3/13/13 7:30 PM: --- That seems to exist in trunk only, so maybe the reference should just be removed from book.xml, or we add security.xml in 0.94 too. was (Author: lhofhansl): That seems to exist in trunk only. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601534#comment-13601534 ] Lars Hofhansl commented on HBASE-8088: -- The same appears to be try for case_studies.xml, community.xml, and zookeeper.xml. Should those not be copied for some other reason? > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8036) ProtobufUtil.multi behavior is inconsistent in case of errors
[ https://issues.apache.org/jira/browse/HBASE-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601544#comment-13601544 ] Hudson commented on HBASE-8036: --- Integrated in hbase-0.95 #70 (See [https://builds.apache.org/job/hbase-0.95/70/]) HBASE-8036 ProtobufUtil.multi behavior is inconsistent in case of errors (Revision 1456064) Result = FAILURE enis : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java > ProtobufUtil.multi behavior is inconsistent in case of errors > - > > Key: HBASE-8036 > URL: https://issues.apache.org/jira/browse/HBASE-8036 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-8036_v1.patch, hbase-8036_v2.patch, > hbase-8036_v3.patch, hbase-8036_v4.patch > > > ProtobufUtil splits operations by regions and performs multiple client.multi > calls. In case if there are certain errors inside RS, HRegionServer adds the > corresponding exceptions to MultiResponse, PU continues the multi request for > other regions, and returns partial failure. > In case of other errors (for example, region not served exception), the > entire multi operation stops executing, and previous successes and partial > results are disregarded. > ProtobufUtil should probably catch ServiceException separately for each > client.multi call, make it a partial-failure exception for all actions for > this region, and also continue the batch, to make the behavior consistent. > Alternatively, if we want to avoid continuing the batch in case of some > server-wide errors/connection problems/etc., server should do that for > region-specific errors (add exception to results for each action). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8081) Backport HBASE-7213 (separate hlog for meta tables) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601543#comment-13601543 ] Lars Hofhansl commented on HBASE-8081: -- This is definitely a valuable addition to 0.94. At the same it is indeed a pretty radical change. +0.5 :) I would feel better if this had seen more real-life testing. > Backport HBASE-7213 (separate hlog for meta tables) to 0.94 > --- > > Key: HBASE-8081 > URL: https://issues.apache.org/jira/browse/HBASE-8081 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.5 >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.94.7 > > Attachments: 7213-0.94-2.patch, 7213-0.94-3.patch, 7213-0.94.patch > > > I am interested in backporting HBASE-7213 to 0.94. Helps to address more of > the MTTR story. Offline discussion with Lars indicated he is interested as > well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601582#comment-13601582
]
Ted Yu commented on HBASE-8085:
---
+1 on patch
{code}
Running org.apache.hadoop.hbase.io.hfile.TestHFileBlock
2013-03-13 13:11:28.397 java[47431:1203] Unable to load realm info from
SCDynamicStore
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.977 sec
Running org.apache.hadoop.hbase.io.hfile.TestHFileBlockCompatibility
2013-03-13 13:12:15.618 java[47433:1203] Unable to load realm info from
SCDynamicStore
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.114 sec
Running org.apache.hadoop.hbase.util.TestBytes
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.84 sec
{code}
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8040) Race condition in AM after HBASE-7521 (only 0.94)
[ https://issues.apache.org/jira/browse/HBASE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601620#comment-13601620 ] Sergey Shelukhin commented on HBASE-8040: - Hi. Is there plan to commit this? Thanks > Race condition in AM after HBASE-7521 (only 0.94) > - > > Key: HBASE-8040 > URL: https://issues.apache.org/jira/browse/HBASE-8040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.94.7 > > Attachments: HBASE-8040_1.patch, HBASE-8040.patch > > > This is a problem that introduced when we tried to solve HBASE-7521. > https://issues.apache.org/jira/browse/HBASE-7521?focusedCommentId=13576083&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13576083 > See the above comment and exactly the same has happened. Will come up with a > solution for the same. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601622#comment-13601622 ] Enis Soztutar commented on HBASE-7938: -- bq. I considered using a reflection loop to execute all of the tests under this scenario No need to do fancy stuff. We can just call the tests one-by-one within run(). bq. This looks like the IntegrationTestingUtility didn't clean up the test directory after a run. Did you kill or have an error on a previous run? It might be because, two of the tests use the same output dir. > Add integration test for ImportTsv/LoadIncrementalHFiles workflow > - > > Key: HBASE-7938 > URL: https://issues.apache.org/jira/browse/HBASE-7938 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0, 0.98.0 > > Attachments: > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch > > > We have existing unit tests for smoke-testing the packaged MR jobs, however > they do not create a runtime environment that is true to running on a real MR > cluster. This is particularly true in regard to classpaths (HBASE-7934) but > also other static state (HBASE-4802). An integration test that can be pointed > to run on a pseudo-distributed Hadoop deployed on localhost would find these > kinds of problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601632#comment-13601632 ] Nick Dimiduk commented on HBASE-7938: - bq. It might be because, two of the tests use the same output dir. I wonder why I don't see that running locally with a pseudo-distributed cluster. I guess I assumed the test util did more for me than it does. I'll address this with the next patch. > Add integration test for ImportTsv/LoadIncrementalHFiles workflow > - > > Key: HBASE-7938 > URL: https://issues.apache.org/jira/browse/HBASE-7938 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0, 0.98.0 > > Attachments: > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch > > > We have existing unit tests for smoke-testing the packaged MR jobs, however > they do not create a runtime environment that is true to running on a real MR > cluster. This is particularly true in regard to classpaths (HBASE-7934) but > also other static state (HBASE-4802). An integration test that can be pointed > to run on a pseudo-distributed Hadoop deployed on localhost would find these > kinds of problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
Sergey Shelukhin created HBASE-8092: --- Summary: bulk assignment in 0.94 doesn't handle ZK errors very well Key: HBASE-8092 URL: https://issues.apache.org/jira/browse/HBASE-8092 Project: HBase Issue Type: Bug Affects Versions: 0.94.6 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.94.7 It can abort the master if node already exists, and as far as I see, if exists check fails it will get stuck forever (the latter I haven't seen though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8031) Adopt goraci as an Integration test
[
https://issues.apache.org/jira/browse/HBASE-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601634#comment-13601634
]
Hudson commented on HBASE-8031:
---
Integrated in HBase-TRUNK #3955 (See
[https://builds.apache.org/job/HBase-TRUNK/3955/])
HBASE-8031 Adopt goraci as an Integration test (Revision 1456078)
Result = FAILURE
enis :
Files :
*
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
> Adopt goraci as an Integration test
> ---
>
> Key: HBASE-8031
> URL: https://issues.apache.org/jira/browse/HBASE-8031
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.0, 0.98.0, 0.94.7
>
> Attachments: hbase-8031_v1.patch
>
>
> As you might know, I am a big fan of the goraci test that Keith Turner has
> developed, which in turn is inspired by the Accumulo test called Continuous
> Ingest.
> As much as I hate to say it, having to rely on gora and and external github
> library makes using this lib cumbersome. And lately we had to use this for
> testing against secure clusters and with Hadoop2, which gora does not support
> for now.
> So, I am proposing we add this test as an IT in the HBase code base so that
> all HBase devs can benefit from it.
> The original source code can be found here:
> * https://github.com/keith-turner/goraci
> * https://github.com/enis/goraci/
> From the javadoc:
> {code}
> Apache Accumulo [0] has a simple test suite that verifies that data is not
> * lost at scale. This test suite is called continuous ingest. This test runs
> * many ingest clients that continually create linked lists containing 25
> * million nodes. At some point the clients are stopped and a map reduce job
> is
> * run to ensure no linked list has a hole. A hole indicates data was lost.··
> *
> * The nodes in the linked list are random. This causes each linked list to
> * spread across the table. Therefore if one part of a table loses data, then
> it
> * will be detected by references in another part of the table.
> *
> Below is rough sketch of how data is written. For specific details look at
> * the Generator code.
> *
> * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that
> * reference previous million· 4 If this is the 25th set of 1 million nodes,
> * then update 1st set of million to point to last· 5 goto 1
> *
> * The key is that nodes only reference flushed nodes. Therefore a node should
> * never reference a missing node, even if the ingest client is killed at any
> * point in time.
> *
> * Some ASCII art time:
> * [ . . . ] represents one batch of random longs of length WIDTH
> *
> *_
> * | __ |
> * | | ||
> * __+_+_ ||
> * v v v |||
> * first = [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * prev= [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * current = [ . . . . . . . . . . . ] |||
> * |||
> * ... |||
> * |||
> * last= [ . . . . . . . . . . . ] |||
> * | | | | | | | | | | |-|||
> * | |||
> * |___|
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8036) ProtobufUtil.multi behavior is inconsistent in case of errors
[ https://issues.apache.org/jira/browse/HBASE-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601635#comment-13601635 ] Hudson commented on HBASE-8036: --- Integrated in HBase-TRUNK #3955 (See [https://builds.apache.org/job/HBase-TRUNK/3955/]) HBASE-8036 ProtobufUtil.multi behavior is inconsistent in case of errors (Revision 1456063) Result = FAILURE enis : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java > ProtobufUtil.multi behavior is inconsistent in case of errors > - > > Key: HBASE-8036 > URL: https://issues.apache.org/jira/browse/HBASE-8036 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-8036_v1.patch, hbase-8036_v2.patch, > hbase-8036_v3.patch, hbase-8036_v4.patch > > > ProtobufUtil splits operations by regions and performs multiple client.multi > calls. In case if there are certain errors inside RS, HRegionServer adds the > corresponding exceptions to MultiResponse, PU continues the multi request for > other regions, and returns partial failure. > In case of other errors (for example, region not served exception), the > entire multi operation stops executing, and previous successes and partial > results are disregarded. > ProtobufUtil should probably catch ServiceException separately for each > client.multi call, make it a partial-failure exception for all actions for > this region, and also continue the batch, to make the behavior consistent. > Alternatively, if we want to avoid continuing the batch in case of some > server-wide errors/connection problems/etc., server should do that for > region-specific errors (add exception to results for each action). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8092: Attachment: HBASE-8092-v0.patch > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601655#comment-13601655 ] Sergey Shelukhin commented on HBASE-8092: - Here's the patch. Should be 0.94 specific. I think the loop wait should also be replaced with something but this is out of the scope of this change. [~ram_krish] can you please take a look? [~lhofhansl] is this good to fix? Thanks! > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8092: Status: Patch Available (was: Open) > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[
https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601664#comment-13601664
]
Hadoop QA commented on HBASE-8092:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573583/HBASE-8092-v0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4800//console
This message is automatically generated.
> bulk assignment in 0.94 doesn't handle ZK errors very well
> --
>
> Key: HBASE-8092
> URL: https://issues.apache.org/jira/browse/HBASE-8092
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.6
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 0.94.7
>
> Attachments: HBASE-8092-v0.patch
>
>
> It can abort the master if node already exists, and as far as I see, if
> exists check fails it will get stuck forever (the latter I haven't seen
> though).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8093) Change the .META. table name for file system incompatibility
Enis Soztutar created HBASE-8093:
Summary: Change the .META. table name for file system
incompatibility
Key: HBASE-8093
URL: https://issues.apache.org/jira/browse/HBASE-8093
Project: HBase
Issue Type: Improvement
Components: master, Region Assignment, regionserver
Affects Versions: 0.95.0, 0.98.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.95.0, 0.98.0
Having a table named ".META." causes problems on NTFS file system under
windows, since it does not allow file names ending with ".".
Nicolas and I have been thinking of just changing the name of the META to be
{noformat}-META-{noformat}
Since 0.96 is the singularity, and we already moved ROOT to zk, now seems like
a good time to do such a change. There will be no rolling upgrade support.
We can migrate META, and can keep BC, by having an internal alias between the
old meta name, and new meta name. This way, outside scripts and code who reads
META can still work.
See also HBASE-6818, HBASE-6821.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8056) allow StoreScanner to drop deletes from some part of the compaction range
[ https://issues.apache.org/jira/browse/HBASE-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601703#comment-13601703 ] Hudson commented on HBASE-8056: --- Integrated in hbase-0.95 #71 (See [https://builds.apache.org/job/hbase-0.95/71/]) HBASE-8056 allow StoreScanner to drop deletes from some part of the compaction range (Revision 1456108) Result = SUCCESS sershe : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java > allow StoreScanner to drop deletes from some part of the compaction range > - > > Key: HBASE-8056 > URL: https://issues.apache.org/jira/browse/HBASE-8056 > Project: HBase > Issue Type: Task > Components: Compaction, Scanners >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-8056-v0.patch, HBASE-8056-v1.patch, > HBASE-8056-v1.patch > > > Allow StoreScanner to drop deletes from some part of the compaction range. > Needed for stripe compactor, and maybe level compactor (although at present I > am not sure how level compactor will drop deletes at all). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8031) Adopt goraci as an Integration test
[
https://issues.apache.org/jira/browse/HBASE-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601704#comment-13601704
]
Hudson commented on HBASE-8031:
---
Integrated in hbase-0.95 #71 (See
[https://builds.apache.org/job/hbase-0.95/71/])
HBASE-8031 Adopt goraci as an Integration test (Revision 1456079)
Result = SUCCESS
enis :
Files :
*
/hbase/branches/0.95/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
> Adopt goraci as an Integration test
> ---
>
> Key: HBASE-8031
> URL: https://issues.apache.org/jira/browse/HBASE-8031
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.0, 0.98.0, 0.94.7
>
> Attachments: hbase-8031_v1.patch
>
>
> As you might know, I am a big fan of the goraci test that Keith Turner has
> developed, which in turn is inspired by the Accumulo test called Continuous
> Ingest.
> As much as I hate to say it, having to rely on gora and and external github
> library makes using this lib cumbersome. And lately we had to use this for
> testing against secure clusters and with Hadoop2, which gora does not support
> for now.
> So, I am proposing we add this test as an IT in the HBase code base so that
> all HBase devs can benefit from it.
> The original source code can be found here:
> * https://github.com/keith-turner/goraci
> * https://github.com/enis/goraci/
> From the javadoc:
> {code}
> Apache Accumulo [0] has a simple test suite that verifies that data is not
> * lost at scale. This test suite is called continuous ingest. This test runs
> * many ingest clients that continually create linked lists containing 25
> * million nodes. At some point the clients are stopped and a map reduce job
> is
> * run to ensure no linked list has a hole. A hole indicates data was lost.··
> *
> * The nodes in the linked list are random. This causes each linked list to
> * spread across the table. Therefore if one part of a table loses data, then
> it
> * will be detected by references in another part of the table.
> *
> Below is rough sketch of how data is written. For specific details look at
> * the Generator code.
> *
> * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that
> * reference previous million· 4 If this is the 25th set of 1 million nodes,
> * then update 1st set of million to point to last· 5 goto 1
> *
> * The key is that nodes only reference flushed nodes. Therefore a node should
> * never reference a missing node, even if the ingest client is killed at any
> * point in time.
> *
> * Some ASCII art time:
> * [ . . . ] represents one batch of random longs of length WIDTH
> *
> *_
> * | __ |
> * | | ||
> * __+_+_ ||
> * v v v |||
> * first = [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * prev= [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * current = [ . . . . . . . . . . . ] |||
> * |||
> * ... |||
> * |||
> * last= [ . . . . . . . . . . . ] |||
> * | | | | | | | | | | |-|||
> * | |||
> * |___|
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8080) refactor default compactor to make its parts easier to reuse
[ https://issues.apache.org/jira/browse/HBASE-8080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8080: Attachment: HBASE-8080-v1.patch Some feedback from r > refactor default compactor to make its parts easier to reuse > > > Key: HBASE-8080 > URL: https://issues.apache.org/jira/browse/HBASE-8080 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-8080-v0.patch, HBASE-8080-v1.patch > > > Refactor default compactor to make its parts easier to reuse. To make > eventual HBASE-7967 patch smaller. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8034) record on-disk data size for store file and make it available during writing
[ https://issues.apache.org/jira/browse/HBASE-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601711#comment-13601711 ] Sergey Shelukhin commented on HBASE-8034: - Given that Ted and Nick +1d, and the condition for stack's conditional +1 appears to be satisifed I will commit evening-ish if there are no objections. > record on-disk data size for store file and make it available during writing > > > Key: HBASE-8034 > URL: https://issues.apache.org/jira/browse/HBASE-8034 > Project: HBase > Issue Type: Task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Attachments: HBASE-8034-v0.patch, HBASE-8034-v1.patch, > HBASE-8034-v2.patch > > > To better estimate the size of data in the file, and to be able to split > files intelligently during any multi-file compactor like stripe or level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8094) TestTableInputFormatScan doesn't assert anything
Nick Dimiduk created HBASE-8094:
---
Summary: TestTableInputFormatScan doesn't assert anything
Key: HBASE-8094
URL: https://issues.apache.org/jira/browse/HBASE-8094
Project: HBase
Issue Type: Sub-task
Components: mapreduce, test
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
This test makes assertions from within the Reducer. These tests are not
respected because the control harness asserts only that jobs complete, not that
they succeed.
Verify this is true by adding a failure to the reducer:
{noformat}
$ git diff
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
index bab9633..322cb4f 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
@@ -160,6 +160,7 @@ public class TestTableInputFormatScan {
if (lastRow != null && lastRow.length() > 0) {
assertEquals(lastRow, last);
}
+ assertTrue(false);
}
}
{noformat}
The test still passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8094) TestTableInputFormatScan doesn't assert anything
[
https://issues.apache.org/jira/browse/HBASE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-8094:
Attachment: 0001-HBASE-8094-fix-broken-TestTableInputFormatScan.patch
Test for job success, not just job completion.
> TestTableInputFormatScan doesn't assert anything
>
>
> Key: HBASE-8094
> URL: https://issues.apache.org/jira/browse/HBASE-8094
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.0, 0.98.0
>
> Attachments: 0001-HBASE-8094-fix-broken-TestTableInputFormatScan.patch
>
>
> This test makes assertions from within the Reducer. These tests are not
> respected because the control harness asserts only that jobs complete, not
> that they succeed.
> Verify this is true by adding a failure to the reducer:
> {noformat}
> $ git diff
> diff --git
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
>
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> index bab9633..322cb4f 100644
> ---
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> +++
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> @@ -160,6 +160,7 @@ public class TestTableInputFormatScan {
>if (lastRow != null && lastRow.length() > 0) {
> assertEquals(lastRow, last);
>}
> + assertTrue(false);
> }
>
>}
> {noformat}
> The test still passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8094) TestTableInputFormatScan doesn't assert anything
[
https://issues.apache.org/jira/browse/HBASE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-8094:
Status: Patch Available (was: Open)
> TestTableInputFormatScan doesn't assert anything
>
>
> Key: HBASE-8094
> URL: https://issues.apache.org/jira/browse/HBASE-8094
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.0, 0.98.0
>
> Attachments: 0001-HBASE-8094-fix-broken-TestTableInputFormatScan.patch
>
>
> This test makes assertions from within the Reducer. These tests are not
> respected because the control harness asserts only that jobs complete, not
> that they succeed.
> Verify this is true by adding a failure to the reducer:
> {noformat}
> $ git diff
> diff --git
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
>
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> index bab9633..322cb4f 100644
> ---
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> +++
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> @@ -160,6 +160,7 @@ public class TestTableInputFormatScan {
>if (lastRow != null && lastRow.length() > 0) {
> assertEquals(lastRow, last);
>}
> + assertTrue(false);
> }
>
>}
> {noformat}
> The test still passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7835) Implementation of the log splitting without creating intermediate files
[ https://issues.apache.org/jira/browse/HBASE-7835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601728#comment-13601728 ] Jeffrey Zhong commented on HBASE-7835: -- [~saint@gmail.com], [~ted_yu] Are you guys ready to give the magic "+1" or do you need more time to review? Thanks, -Jeffrey > Implementation of the log splitting without creating intermediate files > > > Key: HBASE-7835 > URL: https://issues.apache.org/jira/browse/HBASE-7835 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.95.0 > > Attachments: hbase-7006_12.patch, hbase-7006_1.patch, > hbase-7006_2.patch, hbase-7006_3.patch, hbase-7006_3.patch, > hbase-7006_6.patch, hbase-7006_8.patch > > > The sub task is to check in a separate patch for major implementation for the > proposal. > More checkins will include a new "replay" command and more metrics support. > Thanks, > -Jeffrey -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7847) Use zookeeper multi to clear znodes
[ https://issues.apache.org/jira/browse/HBASE-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7847: -- Attachment: 7847-v1.txt Work in progress > Use zookeeper multi to clear znodes > --- > > Key: HBASE-7847 > URL: https://issues.apache.org/jira/browse/HBASE-7847 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu > Attachments: 7847-v1.txt > > > In ZKProcedureUtil, clearChildZNodes() and clearZNodes(String procedureName) > should utilize zookeeper multi so that they're atomic -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8094) TestTableInputFormatScan doesn't assert anything
[
https://issues.apache.org/jira/browse/HBASE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601734#comment-13601734
]
Enis Soztutar commented on HBASE-8094:
--
+1. Will commit if qa pass.
> TestTableInputFormatScan doesn't assert anything
>
>
> Key: HBASE-8094
> URL: https://issues.apache.org/jira/browse/HBASE-8094
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.0, 0.98.0
>
> Attachments: 0001-HBASE-8094-fix-broken-TestTableInputFormatScan.patch
>
>
> This test makes assertions from within the Reducer. These tests are not
> respected because the control harness asserts only that jobs complete, not
> that they succeed.
> Verify this is true by adding a failure to the reducer:
> {noformat}
> $ git diff
> diff --git
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
>
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> index bab9633..322cb4f 100644
> ---
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> +++
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> @@ -160,6 +160,7 @@ public class TestTableInputFormatScan {
>if (lastRow != null && lastRow.length() > 0) {
> assertEquals(lastRow, last);
>}
> + assertTrue(false);
> }
>
>}
> {noformat}
> The test still passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
