[jira] [Updated] (HBASE-6134) Speed up distribued split log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-6134: -- Fix Version/s: 0.96.0 > Speed up distribued split log > - > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6133) TestRestartCluster failing in 0.92
[ https://issues.apache.org/jira/browse/HBASE-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285453#comment-13285453 ] Hudson commented on HBASE-6133: --- Integrated in HBase-0.92 #430 (See [https://builds.apache.org/job/HBase-0.92/430/]) HBASE-6133 TestRestartCluster failing in 0.92 (Revision 1344096) Result = SUCCESS stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java > TestRestartCluster failing in 0.92 > -- > > Key: HBASE-6133 > URL: https://issues.apache.org/jira/browse/HBASE-6133 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.92.2, 0.94.1 > > Attachments: waitonmaster.txt > > > This test failed a few times just now on 0.92 branch. Seems pretty basic > failure. The master is not up yet and its throwing PleaseHoldException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6088) Region splitting not happened for long time due to ZK exception while creating RS_ZK_SPLITTING node
[
https://issues.apache.org/jira/browse/HBASE-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285472#comment-13285472
]
Hudson commented on HBASE-6088:
---
Integrated in HBase-0.94 #230 (See
[https://builds.apache.org/job/HBase-0.94/230/])
HBASE-6088 Addendum fixes testSplitBeforeSettingSplittingInZK (Ram)
(Revision 1344113)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
> Region splitting not happened for long time due to ZK exception while
> creating RS_ZK_SPLITTING node
>
>
> Key: HBASE-6088
> URL: https://issues.apache.org/jira/browse/HBASE-6088
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: rajeshbabu
> Fix For: 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6088_92.patch, HBASE-6088_94.patch,
> HBASE-6088_94_2.patch, HBASE-6088_94_3.patch, HBASE-6088_trunk.patch,
> HBASE-6088_trunk_2.patch, HBASE-6088_trunk_3.patch, HBASE-6088_trunk_4.patch,
> addendum_6088_94.patch
>
>
> Region splitting not happened for long time due to ZK exception while
> creating RS_ZK_SPLITTING node
> {noformat}
> 2012-05-24 01:45:41,363 INFO org.apache.zookeeper.ClientCnxn: Client session
> timed out, have not heard from server in 26668ms for sessionid
> 0x1377a75f41d0012, closing socket connection and attempting reconnect
> 2012-05-24 01:45:41,464 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
> = ConnectionLoss for /hbase/unassigned/bd1079bf948c672e493432020dc0e144
> {noformat}
> {noformat}
> 2012-05-24 01:45:43,300 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog:
> cleanupCurrentWriter waiting for transactions to get synced total 189377
> synced till here 189365
> 2012-05-24 01:45:48,474 INFO
> org.apache.hadoop.hbase.regionserver.SplitRequest: Running rollback/cleanup
> of failed split of
> ufdr,011365398471659,1337823505339.bd1079bf948c672e493432020dc0e144.; Failed
> setting SPLITTING znode on
> ufdr,011365398471659,1337823505339.bd1079bf948c672e493432020dc0e144.
> java.io.IOException: Failed setting SPLITTING znode on
> ufdr,011365398471659,1337823505339.bd1079bf948c672e493432020dc0e144.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:242)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:450)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.zookeeper.KeeperException$BadVersionException:
> KeeperErrorCode = BadVersion for
> /hbase/unassigned/bd1079bf948c672e493432020dc0e144
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:115)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1246)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.setData(RecoverableZooKeeper.java:321)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:659)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.transitionNode(ZKAssign.java:811)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.transitionNode(ZKAssign.java:747)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.transitionNodeSplitting(SplitTransaction.java:919)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.createNodeSplitting(SplitTransaction.java:869)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:239)
> ... 5 more
> 2012-05-24 01:45:48,476 INFO
> org.apache.hadoop.hbase.regionserver.SplitRequest: Successful rollback of
> failed split of
> ufdr,011365398471659,1337823505339.bd1079bf948c672e493432020dc0e144.
> {noformat}
> {noformat}
> 2012-05-24 01:47:28,141 ERROR
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node
> /hbase/unassigned/bd1079bf948c672e493432020dc0e144 already exists and this is
> not a retry
> 2012-05-24 01:47:28,142 INFO
> org.apache.hadoop.hbase.regionserver.SplitRequest: Running rollback/cleanup
> of failed split of
> ufdr,011365398471659,1337823505339.bd1079bf948c672e493432020dc0e144.; Failed
> create of ephemeral /hbase/unassigned/bd1079bf948c672e493432020d
[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect
[ https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285487#comment-13285487 ] Lars Hofhansl commented on HBASE-5974: -- W.r.t. to RegionScannerHolder. I see that makes sense. As long as the sequence numbering also works when the CP intercepts a scanner and provides its own implementation. I don't think there'd be a case where a CP would want to interfere with the sequence numbers. Re: the NoSuchMethodException check. I was wondering if there is anything with RPC version numbering to deal with this instead. > Scanner retry behavior with RPC timeout on next() seems incorrect > - > > Key: HBASE-5974 > URL: https://issues.apache.org/jira/browse/HBASE-5974 > Project: HBase > Issue Type: Bug > Components: client, regionserver >Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0 >Reporter: Todd Lipcon >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.94.1 > > Attachments: HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch > > > I'm seeing the following behavior: > - set RPC timeout to a short value > - call next() for some batch of rows, big enough so the client times out > before the result is returned > - the HConnectionManager stuff will retry the next() call to the same server. > At this point, one of two things can happen: 1) the previous next() call will > still be processing, in which case you get a LeaseException, because it was > removed from the map during the processing, or 2) the next() call will > succeed but skip the prior batch of rows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5993) Add a no-read Append
[ https://issues.apache.org/jira/browse/HBASE-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285493#comment-13285493 ] Lars Hofhansl commented on HBASE-5993: -- Honestly, I still do not understand what Jacques is proposing. In order to append to something you'd have to read that something first. HBase has no in place updates (for a good reason). So one could: # Replace the KV if it is still in the memstore. # Store incremental changes (somewhere?) and combine upon read from HBase. > Add a no-read Append > > > Key: HBASE-5993 > URL: https://issues.apache.org/jira/browse/HBASE-5993 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Jacques >Priority: Critical > > HBASE-4102 added an atomic append. For high performance situations, it would > be helpful to be able to do appends that don't actually require a read of the > existing value. This would be useful in building a growing set of values. > Our original use case was for implementing a form of search in HBase where a > cell would contain a list of document ids associated with a particular > keyword for search. However it seems like it would also be useful to provide > substantial performance improvements for most Append scenarios. > Within the client API, the simplest way to implement this would be to > leverage the existing Append api. If the Append is marked as > setReturnResults(false), use this code path. If result return is requested, > use the existing Append implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6134) Speed up distribued split log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-6134: Attachment: HBASE-6134.patch > Speed up distribued split log > - > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Speed up distribued split log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285496#comment-13285496 ] chunhui shen commented on HBASE-6134: - @ted In my opition, we could do the improvement for split-worker. In the patch , I change the split logic which executed by the split-worker same as local-master-split. It means, there are multi threads to creater writer and append hlog entry. Also, we found closing writer is quite slow, so make the closeStreams() executed by multi threads. With the patch, we test the time of distribued split log again, and reduce the time from 165s+ to 38s+ (34hlog files 4 regionervers) > Speed up distribued split log > - > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285498#comment-13285498 ] chunhui shen commented on HBASE-6134: - Review Board https://reviews.apache.org/r/5294/ > Improvement for split-worker to speed up distributed-split-log > -- > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-6134: Summary: Improvement for split-worker to speed up distributed-split-log (was: Speed up distribued split log) > Improvement for split-worker to speed up distributed-split-log > -- > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285502#comment-13285502 ] ramkrishna.s.vasudevan commented on HBASE-6134: --- @Chunhui Nice work. bq.Also, we found closing writer is quite slow Any reason why it was slow? > Improvement for split-worker to speed up distributed-split-log > -- > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6116) Allow parallel HDFS writes for HLogs.
[ https://issues.apache.org/jira/browse/HBASE-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285503#comment-13285503 ] Lars Hofhansl commented on HBASE-6116: -- Waiting for HDFS-1783 API to stabilize before I post a patch here. I don't currently have access to a real cluster, so if someone could do some testing in a real cluster with Hadoop-Trunk and HBase-Trunk, please let me know! > Allow parallel HDFS writes for HLogs. > - > > Key: HBASE-6116 > URL: https://issues.apache.org/jira/browse/HBASE-6116 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > > In HDFS-1783 I adapted Dhrubas changes to be used in Hadoop trunk. > This issue will include the necessary reflection changes to optionally enable > this for the WALs in HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5993) Add a no-read Append
[ https://issues.apache.org/jira/browse/HBASE-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285504#comment-13285504 ] Jieshan Bean commented on HBASE-5993: - One KeyValue will include 2 parts: The original value, and the appended value. So if we append new value, just add this value into the "additional" part(Maybe we can implement this by introducing a new KeyValue.Type). So we need to combine the values. (@Jacques: Please correct me if I misunderstood anything). I'm wondering whether it is really necessary. > Add a no-read Append > > > Key: HBASE-5993 > URL: https://issues.apache.org/jira/browse/HBASE-5993 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Jacques >Priority: Critical > > HBASE-4102 added an atomic append. For high performance situations, it would > be helpful to be able to do appends that don't actually require a read of the > existing value. This would be useful in building a growing set of values. > Our original use case was for implementing a form of search in HBase where a > cell would contain a list of document ids associated with a particular > keyword for search. However it seems like it would also be useful to provide > substantial performance improvements for most Append scenarios. > Within the client API, the simplest way to implement this would be to > leverage the existing Append api. If the Append is marked as > setReturnResults(false), use this code path. If result return is requested, > use the existing Append implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285511#comment-13285511 ] chunhui shen commented on HBASE-6134: - @ram Through the log, we found writer.close() took about 30ms~50ms. So, it will took about 20s if there are 400 regions in one hlog file. > Improvement for split-worker to speed up distributed-split-log > -- > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[
https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285513#comment-13285513
]
chunhui shen commented on HBASE-6134:
-
@ram
Could you take a look at your environment, hom much time writer.close() took?
Through the log:
LOG.info("Closed path " + wap.p + " (wrote " + wap.editsWritten
+ " edits in " + (wap.nanosSpent / 1000 / 1000) + "ms)");
> Improvement for split-worker to speed up distributed-split-log
> --
>
> Key: HBASE-6134
> URL: https://issues.apache.org/jira/browse/HBASE-6134
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6134.patch
>
>
> First，we do the test between local-master-split and distributed split log
> Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths
> splitting work)
> local-master-split:60s+
> distributed-split-log:165s+
> In fact, in our production environment, distributed-split-log also took 60s
> with 30 regionservers for 34 hlog files (regionserver may be in high load)
> We found split-worker split one log file took about 20s.
> I think we should do the improvement for this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285515#comment-13285515 ] ramkrishna.s.vasudevan commented on HBASE-6134: --- Ok. I will check that but may not be today. :) > Improvement for split-worker to speed up distributed-split-log > -- > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6114) CacheControl flags should be tunable per table schema per CF
[ https://issues.apache.org/jira/browse/HBASE-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285524#comment-13285524 ] Hudson commented on HBASE-6114: --- Integrated in HBase-0.92 #431 (See [https://builds.apache.org/job/HBase-0.92/431/]) HBASE-6114. CacheControl flags should be tunable per table schema per CF (Revision 1344107) Result = SUCCESS apurtell : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCacheOnWriteInSchema.java > CacheControl flags should be tunable per table schema per CF > > > Key: HBASE-6114 > URL: https://issues.apache.org/jira/browse/HBASE-6114 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: 6114-0.92-v2.patch, 6114-0.92.patch, 6114-0.94-v2.patch, > 6114-0.94.patch, 6114-trunk-v2.patch, 6114-trunk.patch, 6114.patch > > > CacheControl flags should be tunable per table schema per CF, especially > cacheDataOnWrite, also cacheIndexesOnWrite and cacheBloomsOnWrite. > It looks like Store uses CacheConfig(Configuration conf, HColumnDescriptor > family) to construct the CacheConfig, so it's a simple change there to > override configuration properties with values of table schema attributes if > present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect
[ https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285532#comment-13285532 ] Anoop Sam John commented on HBASE-5974: --- @Lars Or is it this kind of a compatibility is not needed? Only upgradation at client side and not at server. I am not sure what kind of compatibility we need to maintain across new minor versions (from 94.0 to 94.1) Also see the comments for the issue HBASE-6124. [Similar to our case] https://issues.apache.org/jira/browse/HBASE-6124?focusedCommentId=13285008&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13285008 https://issues.apache.org/jira/browse/HBASE-6124?focusedCommentId=13285016&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13285016 > Scanner retry behavior with RPC timeout on next() seems incorrect > - > > Key: HBASE-5974 > URL: https://issues.apache.org/jira/browse/HBASE-5974 > Project: HBase > Issue Type: Bug > Components: client, regionserver >Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0 >Reporter: Todd Lipcon >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.94.1 > > Attachments: HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch > > > I'm seeing the following behavior: > - set RPC timeout to a short value > - call next() for some batch of rows, big enough so the client times out > before the result is returned > - the HConnectionManager stuff will retry the next() call to the same server. > At this point, one of two things can happen: 1) the previous next() call will > still be processing, in which case you get a LeaseException, because it was > removed from the map during the processing, or 2) the next() call will > succeed but skip the prior batch of rows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6089) SSH and AM.joinCluster causes Concurrent Modification exception.
[
https://issues.apache.org/jira/browse/HBASE-6089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285541#comment-13285541
]
ramkrishna.s.vasudevan commented on HBASE-6089:
---
Thinking again on this, there are some places where we wait for the
this.regions to be populated.
{code}
{code}
> SSH and AM.joinCluster causes Concurrent Modification exception.
>
>
> Key: HBASE-6089
> URL: https://issues.apache.org/jira/browse/HBASE-6089
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6089_92.patch, HBASE-6089_94.patch,
> HBASE-6089_trunk.patch
>
>
> AM.regions map is parallely accessed in SSH and Master initialization leading
> to ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6089) SSH and AM.joinCluster causes Concurrent Modification exception.
[
https://issues.apache.org/jira/browse/HBASE-6089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285541#comment-13285541
]
ramkrishna.s.vasudevan edited comment on HBASE-6089 at 5/30/12 10:09 AM:
-
I was not clear in the previous comment. Thinking again on this, there are
some places where we wait for the this.regions to be populated.
{code}
synchronized(regions) {
while(!regions.containsKey(regionInfo)) {
// We should receive a notification, but it's
// better to have a timeout to recheck the condition here:
// it lowers the impact of a race condition if any
regions.wait(100);
}
{code}
When we need to update the this.servers also along with this.regions then we
need to have some sync block. I agree with Anoop here.
was (Author: ram_krish):
Thinking again on this, there are some places where we wait for the
this.regions to be populated.
{code}
{code}
> SSH and AM.joinCluster causes Concurrent Modification exception.
>
>
> Key: HBASE-6089
> URL: https://issues.apache.org/jira/browse/HBASE-6089
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
> Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-6089_92.patch, HBASE-6089_94.patch,
> HBASE-6089_trunk.patch
>
>
> AM.regions map is parallely accessed in SSH and Master initialization leading
> to ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5733: -- Fix Version/s: 0.94.1 0.92.2 > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285551#comment-13285551 ] ramkrishna.s.vasudevan commented on HBASE-5733: --- I think its better we commit it to 0.94.1 also before Lars could take the Rc. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reopened HBASE-5733: --- Reopening so that once committed to other versions we can close it. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6096) AccessController v2
[ https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285554#comment-13285554 ] Matteo Bertozzi commented on HBASE-6096: @Laxman if you look at TableAuthManager there're two SkipList USER_CACHE and TABLE_USER_CACHE USER_CACHE contains the global rights, TABLE_USER_CACHE contains the table one. hbase.superuser is read at startup TableAuthManager.initGlobal() and fills the USER_CACHE with specified users with RWCA rights (so is not just ADMIN is RWCA) All CA operation are checked against global rights (USER_CACHE) so specifying CA in table rights has no effect. grant 'user', 'RWCA' -> fill USER_CACHE global rights grant 'user', 'RW', 'table' -> FILL TABLE_USER_CACHE this means if you're "global granted" (CREATE/ADMIN) you can add/modify/remove tables with the restriction that, if you've CREATE you can just modify/delete your own table while ADMIN you can modify/delete all the available tables. if you're table granted you can do just read/write depends on what is your permission, but you don't have the ability to create/modify/delete even if you've CA (CA are not checked against TABLE_USER_CACHE just USER_CACHE) for checkAndPut()/checkAndDelete() there's a jira open HBASE-6062 and yes it needs to check both Read and Write. > AccessController v2 > --- > > Key: HBASE-6096 > URL: https://issues.apache.org/jira/browse/HBASE-6096 > Project: HBase > Issue Type: Umbrella > Components: security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > Umbrella issue for iteration on the initial AccessController drop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5722) NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW down.
[
https://issues.apache.org/jira/browse/HBASE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-5722:
--
Affects Version/s: 0.94.0
Fix Version/s: (was: 0.94.0)
0.94.1
0.96.0
> NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW
> down.
> --
>
> Key: HBASE-5722
> URL: https://issues.apache.org/jira/browse/HBASE-5722
> Project: HBase
> Issue Type: Bug
> Components: zookeeper
>Affects Versions: 0.94.0, 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5722.patch
>
>
> {code}
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> for (String node: nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte [] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> {code}
> The above code can throw NPE when listChildrenAndWatchForNewChildren returns
> null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5722) NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW down.
[
https://issues.apache.org/jira/browse/HBASE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan reopened HBASE-5722:
---
This got committed to 0.96 only and not to 0.94. Hence reopening so that once
committed we can close it.
> NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW
> down.
> --
>
> Key: HBASE-5722
> URL: https://issues.apache.org/jira/browse/HBASE-5722
> Project: HBase
> Issue Type: Bug
> Components: zookeeper
>Affects Versions: 0.94.0, 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5722.patch
>
>
> {code}
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> for (String node: nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte [] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> {code}
> The above code can throw NPE when listChildrenAndWatchForNewChildren returns
> null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6062) preCheckAndPut/Delete() checks for READ when also a WRITE is performed
[ https://issues.apache.org/jira/browse/HBASE-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laxman updated HBASE-6062: -- Component/s: security > preCheckAndPut/Delete() checks for READ when also a WRITE is performed > -- > > Key: HBASE-6062 > URL: https://issues.apache.org/jira/browse/HBASE-6062 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Labels: acl, security > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-6062-v0.patch > > > preCheckAndPut() and preCheckAndDelete() checks for READ when they also want > to WRITE... > for me checking for WRITE permission is the right thing... > what do you say about that? keep READ, replace with WRITE? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-6134: -- Status: Patch Available (was: Open) > Improvement for split-worker to speed up distributed-split-log > -- > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6083) Modify old filter tests to use Junit4/no longer use HBaseTestCase
[ https://issues.apache.org/jira/browse/HBASE-6083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285604#comment-13285604 ] Hudson commented on HBASE-6083: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #32 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/32/]) HBASE-6083 Modify old filter tests to use Junit4/no longer use HBaseTestCase (Revision 1344111) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestBitComparator.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPaginationFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilterList.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestInclusiveStopFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestPageFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestPrefixFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestRandomRowFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestSingleColumnValueExcludeFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestSingleColumnValueFilter.java > Modify old filter tests to use Junit4/no longer use HBaseTestCase > - > > Key: HBASE-6083 > URL: https://issues.apache.org/jira/browse/HBASE-6083 > Project: HBase > Issue Type: Improvement >Reporter: Juhani Connolly >Assignee: Juhani Connolly >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6083v2.txt, 6083v2.txt, hbase-6083.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6043) Add Increment Coalescing in thrift.
[ https://issues.apache.org/jira/browse/HBASE-6043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285603#comment-13285603 ] Hudson commented on HBASE-6043: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #32 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/32/]) HBASE-6043 Add Increment Coalescing in thrift. (Revision 1344086) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/thrift/IncrementCoalescer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/thrift/IncrementCoalescerMBean.java > Add Increment Coalescing in thrift. > --- > > Key: HBASE-6043 > URL: https://issues.apache.org/jira/browse/HBASE-6043 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6043-0.patch, HBASE-6043-1.patch, > HBASE-6043-10.patch, HBASE-6043-2.patch, HBASE-6043-3.patch, > HBASE-6043-4.patch, HBASE-6043-5.patch, HBASE-6043-6.patch, > HBASE-6043-7.patch, HBASE-6043-8.patch, HBASE-6043-9.patch, > HBASE-6043-ADD.patch > > > Since the thrift server uses the client api reducing the number of rpc's > greatly speeds up increments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6133) TestRestartCluster failing in 0.92
[ https://issues.apache.org/jira/browse/HBASE-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285606#comment-13285606 ] Hudson commented on HBASE-6133: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #32 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/32/]) HBASE-6133 TestRestartCluster failing in 0.92 (Revision 1344098) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java > TestRestartCluster failing in 0.92 > -- > > Key: HBASE-6133 > URL: https://issues.apache.org/jira/browse/HBASE-6133 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.92.2, 0.94.1 > > Attachments: waitonmaster.txt > > > This test failed a few times just now on 0.92 branch. Seems pretty basic > failure. The master is not up yet and its throwing PleaseHoldException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6114) CacheControl flags should be tunable per table schema per CF
[ https://issues.apache.org/jira/browse/HBASE-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285605#comment-13285605 ] Hudson commented on HBASE-6114: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #32 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/32/]) HBASE-6114. CacheControl flags should be tunable per table schema per CF (Revision 1344105) Result = FAILURE apurtell : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCacheOnWriteInSchema.java > CacheControl flags should be tunable per table schema per CF > > > Key: HBASE-6114 > URL: https://issues.apache.org/jira/browse/HBASE-6114 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: 6114-0.92-v2.patch, 6114-0.92.patch, 6114-0.94-v2.patch, > 6114-0.94.patch, 6114-trunk-v2.patch, 6114-trunk.patch, 6114.patch > > > CacheControl flags should be tunable per table schema per CF, especially > cacheDataOnWrite, also cacheIndexesOnWrite and cacheBloomsOnWrite. > It looks like Store uses CacheConfig(Configuration conf, HColumnDescriptor > family) to construct the CacheConfig, so it's a simple change there to > override configuration properties with values of table schema attributes if > present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6096) AccessController v2
[ https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285611#comment-13285611 ] Laxman commented on HBASE-6096: --- @Matteo, Thanks a lot for detailed explanation. With your above explanation, I understand that current design/implementation ignores TABLE CREATE/ADMIN permissions. But here, I'm trying to get more on "How it should be?" than "How it is currently?" Found related discussion (TABLE ADMIN vs GLOBAL ADMIN) @ HBASE-5372. I'm sorry if I'm asking basic questions. bq. IMO, GLOBAL ADMIN (for all tables) semantics should be inline with TABLE ADMIN (for one table). Is this correct expectation or any different opinions? > AccessController v2 > --- > > Key: HBASE-6096 > URL: https://issues.apache.org/jira/browse/HBASE-6096 > Project: HBase > Issue Type: Umbrella > Components: security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > Umbrella issue for iteration on the initial AccessController drop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6109: --- Attachment: 6109.v19.patch > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6109: --- Status: Patch Available (was: Open) > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[ https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285623#comment-13285623 ] Hadoop QA commented on HBASE-6134: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530177/HBASE-6134.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed org.apache.hadoop.hbase.regionserver.wal.TestWALReplay Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2051//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2051//console This message is automatically generated. > Improvement for split-worker to speed up distributed-split-log > -- > > Key: HBASE-6134 > URL: https://issues.apache.org/jira/browse/HBASE-6134 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6134.patch > > > First，we do the test between local-master-split and distributed split log > Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths > splitting work) > local-master-split:60s+ > distributed-split-log:165s+ > In fact, in our production environment, distributed-split-log also took 60s > with 30 regionservers for 34 hlog files (regionserver may be in high load) > We found split-worker split one log file took about 20s. > I think we should do the improvement for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285640#comment-13285640 ] Hadoop QA commented on HBASE-6109: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530195/6109.v19.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 26 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2052//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2052//console This message is automatically generated. > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285671#comment-13285671 ] nkeywal commented on HBASE-6109: I think it's ok for a commit. >From the code I read, we should have the same behavior as before on split. I >will write some parallel tests later on, but I would expect the same behavior >as today at least. It may take time as I may encounter some flakiness on this >path ;-). I don't have a test class for NotifiableConcurrentSkipListMap, this class is small so I don't think it's an issue right now. I will push one with the other tests I will write. > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-6109: -- Fix Version/s: 0.96.0 > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6012) AssignmentManager#asyncSetOfflineInZooKeeper wouldn't force node offline
[
https://issues.apache.org/jira/browse/HBASE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-6012:
--
Affects Version/s: 0.96.0
Fix Version/s: 0.96.0
> AssignmentManager#asyncSetOfflineInZooKeeper wouldn't force node offline
>
>
> Key: HBASE-6012
> URL: https://issues.apache.org/jira/browse/HBASE-6012
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6012.patch
>
>
> As the javadoc of method and the log message
> {code}
> /**
>* Set region as OFFLINED up in zookeeper asynchronously.
>*/
> boolean asyncSetOfflineInZooKeeper(
> ...
> master.abort("Unexpected ZK exception creating/setting node OFFLINE", e);
> ...
> }
> {code}
> I think AssignmentManager#asyncSetOfflineInZooKeeper should also force node
> offline, just like AssignmentManager#setOfflineInZooKeeper do. Otherwise, it
> may cause bulk assign failed which called this method.
> Error log on the master caused by the issue
> 2012-05-12 01:40:09,437 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=writetest,1YTQDPGLXBTICHOPQ6IL,1336590857771.674da422fc7cb9a7d42c74499ace1d93.
> state=PENDING_CLOSE, ts=1336757876856
> 2012-05-12 01:40:09,437 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x23736bf74780082 Async create of unassigned node for
> 674da422fc7cb9a7d42c74499ace1d93 with OFFLINE state
> 2012-05-12 01:40:09,446 WARN
> org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback:
> rc != 0 for /hbase-func1/unassigned/674da422fc7cb9a7d42c74499ace1d93 --
> retryable connectionloss -- FIX see
> http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A2
> 2012-05-12 01:40:09,447 FATAL org.apache.hadoop.hbase.master.HMaster:
> Connectionloss writing unassigned at
> /hbase-func1/unassigned/674da422fc7cb9a7d42c74499ace1d93, rc=-110
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6012) AssignmentManager#asyncSetOfflineInZooKeeper wouldn't force node offline
[
https://issues.apache.org/jira/browse/HBASE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285698#comment-13285698
]
ramkrishna.s.vasudevan commented on HBASE-6012:
---
I think this change we need to get into trunk after reviews for the SSH part to
work fine introduced in HBASE-5914.
> AssignmentManager#asyncSetOfflineInZooKeeper wouldn't force node offline
>
>
> Key: HBASE-6012
> URL: https://issues.apache.org/jira/browse/HBASE-6012
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6012.patch
>
>
> As the javadoc of method and the log message
> {code}
> /**
>* Set region as OFFLINED up in zookeeper asynchronously.
>*/
> boolean asyncSetOfflineInZooKeeper(
> ...
> master.abort("Unexpected ZK exception creating/setting node OFFLINE", e);
> ...
> }
> {code}
> I think AssignmentManager#asyncSetOfflineInZooKeeper should also force node
> offline, just like AssignmentManager#setOfflineInZooKeeper do. Otherwise, it
> may cause bulk assign failed which called this method.
> Error log on the master caused by the issue
> 2012-05-12 01:40:09,437 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=writetest,1YTQDPGLXBTICHOPQ6IL,1336590857771.674da422fc7cb9a7d42c74499ace1d93.
> state=PENDING_CLOSE, ts=1336757876856
> 2012-05-12 01:40:09,437 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x23736bf74780082 Async create of unassigned node for
> 674da422fc7cb9a7d42c74499ace1d93 with OFFLINE state
> 2012-05-12 01:40:09,446 WARN
> org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback:
> rc != 0 for /hbase-func1/unassigned/674da422fc7cb9a7d42c74499ace1d93 --
> retryable connectionloss -- FIX see
> http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A2
> 2012-05-12 01:40:09,447 FATAL org.apache.hadoop.hbase.master.HMaster:
> Connectionloss writing unassigned at
> /hbase-func1/unassigned/674da422fc7cb9a7d42c74499ace1d93, rc=-110
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics
[ https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6131: - Attachment: 6131_094.txt Here is what I applied to 0.94 branch. > Add attribution for code added by HBASE-5533 metrics > > > Key: HBASE-6131 > URL: https://issues.apache.org/jira/browse/HBASE-6131 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Attachments: 6131.txt, 6131_094.txt > > > See the comment over in > https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920 > The metrics histogram code was copied w/o attribution. Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics
[ https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285735#comment-13285735 ] Hadoop QA commented on HBASE-6131: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530216/6131_094.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2053//console This message is automatically generated. > Add attribution for code added by HBASE-5533 metrics > > > Key: HBASE-6131 > URL: https://issues.apache.org/jira/browse/HBASE-6131 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.94.1 > > Attachments: 6131.txt, 6131_094.txt > > > See the comment over in > https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920 > The metrics histogram code was copied w/o attribution. Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics
[ https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6131: - Resolution: Fixed Fix Version/s: 0.94.1 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.94 and to trunk. Thanks Jeff and Ted for finding this copying. > Add attribution for code added by HBASE-5533 metrics > > > Key: HBASE-6131 > URL: https://issues.apache.org/jira/browse/HBASE-6131 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.94.1 > > Attachments: 6131.txt, 6131_094.txt > > > See the comment over in > https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920 > The metrics histogram code was copied w/o attribution. Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6116) Allow parallel HDFS writes for HLogs.
[ https://issues.apache.org/jira/browse/HBASE-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285760#comment-13285760 ] Andrew Purtell commented on HBASE-6116: --- When there is a patch I can try it out on a test cluster in EC2. > Allow parallel HDFS writes for HLogs. > - > > Key: HBASE-6116 > URL: https://issues.apache.org/jira/browse/HBASE-6116 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > > In HDFS-1783 I adapted Dhrubas changes to be used in Hadoop trunk. > This issue will include the necessary reflection changes to optionally enable > this for the WALs in HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5993) Add a no-read Append
[ https://issues.apache.org/jira/browse/HBASE-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285764#comment-13285764 ] Jacques commented on HBASE-5993: The reason this can make sense is data overhead. In a case where we are capturing a large number of small values, the KeyValue overhead is substantial. The original use case is one where I'm adding to a list of documents that contain a certain term (search index). Let's say that each document number is a four byte int. Right now there are two options: use the existing append which means one will become swamped with reads as the cell value grows over time (this would also wreak havoc on memstore flushes as the cell value become megabytes in size and we're just adding another four bytes once a day). On the flipside, using separate columns creates a substantial amount of overhead for each value added. This utility of this functionality also extends to situations where people are capturing a large sequence of small values in monitoring applications. (Sematext are basically trying to create this functionality already with their HBaseHUT work.) Yes, an additional KeyValue.Type is needed. When this type is read, the return functionality goes and get all the appended values (and the last full value) and then combines them on return. As compactions are done, the complete merged values are created. I'm swamped right now but am going to try to write up a short design doc in the next couple of weeks and get you guys to review my approach since this will have to touch a number of components. I also need to make sure to manage edge cases like what happens if you do a no-read append and no existing value exists (probably ok--even though read back performance will be poor). > Add a no-read Append > > > Key: HBASE-5993 > URL: https://issues.apache.org/jira/browse/HBASE-5993 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Jacques >Priority: Critical > > HBASE-4102 added an atomic append. For high performance situations, it would > be helpful to be able to do appends that don't actually require a read of the > existing value. This would be useful in building a growing set of values. > Our original use case was for implementing a form of search in HBase where a > cell would contain a list of document ids associated with a particular > keyword for search. However it seems like it would also be useful to provide > substantial performance improvements for most Append scenarios. > Within the client API, the simplest way to implement this would be to > leverage the existing Append api. If the Append is marked as > setReturnResults(false), use this code path. If result return is requested, > use the existing Append implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6096) AccessController v2
[ https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285766#comment-13285766 ] Andrew Purtell commented on HBASE-6096: --- bq. hbase.superuser is read at startup TableAuthManager.initGlobal() and fills the USER_CACHE with specified users with RWCA rights (so is not just ADMIN is RWCA) The "superuser" concept is a legacy from the initial implementation. Instead of having ADMIN rights mean something per user, there was/is an implicit grant of ADMIN rights to the superuser and that is it, for simplicity. So in our production the "hbase" user is used to manage the cluster by ops, and users have grants of only READ or WRITE as appropriate. bq. GLOBAL ADMIN (for all tables) semantics should be inline with TABLE ADMIN (for one table). +1 The "superuser" shortcut should be removed. Instead the AccessController could add a GLOBAL ADMIN grant on demand for the service principal of the master and regionservers when creating the ACL table. I don't think anyone is setting the "superuser" to anything other than the service principal. Also we could drop the "owner" concept (and table attribute) and instead have the AccessController add a TABLE ADMIN grant at table creation time, as discussed in HBASE-5372. > AccessController v2 > --- > > Key: HBASE-6096 > URL: https://issues.apache.org/jira/browse/HBASE-6096 > Project: HBase > Issue Type: Umbrella > Components: security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > Umbrella issue for iteration on the initial AccessController drop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6096) AccessController v2
[ https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285769#comment-13285769 ] Andrew Purtell commented on HBASE-6096: --- bq. checkAndPut/checkAndDelete These should need READ + WRITE. > AccessController v2 > --- > > Key: HBASE-6096 > URL: https://issues.apache.org/jira/browse/HBASE-6096 > Project: HBase > Issue Type: Umbrella > Components: security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > Umbrella issue for iteration on the initial AccessController drop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6096) AccessController v2
[ https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285771#comment-13285771 ] Andrew Purtell commented on HBASE-6096: --- And to clarify, the "superuser" can do anything, there is not only an implicit grant of ADMIN. > AccessController v2 > --- > > Key: HBASE-6096 > URL: https://issues.apache.org/jira/browse/HBASE-6096 > Project: HBase > Issue Type: Umbrella > Components: security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > Umbrella issue for iteration on the initial AccessController drop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6096) AccessController v2
[ https://issues.apache.org/jira/browse/HBASE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285771#comment-13285771 ] Andrew Purtell edited comment on HBASE-6096 at 5/30/12 4:02 PM: And to clarify, the "superuser" can do anything, there is not only an implicit grant of ADMIN. Now that people are working on functioning global permissions it's not necessary to have this shortcut. was (Author: apurtell): And to clarify, the "superuser" can do anything, there is not only an implicit grant of ADMIN. > AccessController v2 > --- > > Key: HBASE-6096 > URL: https://issues.apache.org/jira/browse/HBASE-6096 > Project: HBase > Issue Type: Umbrella > Components: security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > Umbrella issue for iteration on the initial AccessController drop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-6109: -- Attachment: 6049-v3.patch Patch v3 rebased on trunk. > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285780#comment-13285780 ] Andrew Purtell commented on HBASE-4720: --- +1 except for small debug logging nit that can be corrected on commit. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.96.0, 0.94.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.patch, HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-6109: -- Attachment: (was: 6049-v3.patch) > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285780#comment-13285780 ] Andrew Purtell edited comment on HBASE-4720 at 5/30/12 4:12 PM: +1 except for small debug logging nit that can be corrected on commit. See https://reviews.apache.org/r/5259/diff/3/?file=110632#file110632line294 was (Author: apurtell): +1 except for small debug logging nit that can be corrected on commit. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.96.0, 0.94.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.patch, HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6049) Serializing "List" containing null elements will cause NullPointerException in HbaseObjectWritable.writeObject()
[ https://issues.apache.org/jira/browse/HBASE-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-6049: -- Attachment: 6049-v3.patch Patch v3 rebased on trunk. > Serializing "List" containing null elements will cause NullPointerException > in HbaseObjectWritable.writeObject() > > > Key: HBASE-6049 > URL: https://issues.apache.org/jira/browse/HBASE-6049 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.94.0 >Reporter: Maryann Xue > Attachments: 6049-v3.patch, HBASE-6049-v2.patch, HBASE-6049-v3.patch, > HBASE-6049.patch > > > An error case could be in Coprocessor AggregationClient, the median() > function handles an empty region and returns a List Object with the first > element as a Null value. NPE occurs in the RPC response stage and the > response never gets sent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4676) Prefix Compression - Trie data block encoding
[ https://issues.apache.org/jira/browse/HBASE-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285793#comment-13285793 ] Jacques commented on HBASE-4676: Random thought on this. I was talking to Nicolas Spiegelberg and he was talking about how they are exploring changing encoding schemes when data becomes more permanent. Don't quote me on this but it sounded like they were considering using gzip compression on major compactions. I was wondering if something similar made sense here. Basically, use less compression the shorter the likely lifetime of files. Use the Trie compressions on only major compactions. This way the performance difference could be less of a serious issue since you're paying for it a fewer number of times. I glanced around and didn't see any Jiras about a two-tiered approach of data storage formats but it seems like that would be a prerequisite for a hybrid/tiered approach. > Prefix Compression - Trie data block encoding > - > > Key: HBASE-4676 > URL: https://issues.apache.org/jira/browse/HBASE-4676 > Project: HBase > Issue Type: New Feature > Components: io, performance, regionserver >Affects Versions: 0.90.6 >Reporter: Matt Corgan >Assignee: Matt Corgan > Attachments: HBASE-4676-0.94-v1.patch, PrefixTrie_Format_v1.pdf, > PrefixTrie_Performance_v1.pdf, SeeksPerSec by blockSize.png, > hbase-prefix-trie-0.1.jar > > > The HBase data block format has room for 2 significant improvements for > applications that have high block cache hit ratios. > First, there is no prefix compression, and the current KeyValue format is > somewhat metadata heavy, so there can be tremendous memory bloat for many > common data layouts, specifically those with long keys and short values. > Second, there is no random access to KeyValues inside data blocks. This > means that every time you double the datablock size, average seek time (or > average cpu consumption) goes up by a factor of 2. The standard 64KB block > size is ~10x slower for random seeks than a 4KB block size, but block sizes > as small as 4KB cause problems elsewhere. Using block sizes of 256KB or 1MB > or more may be more efficient from a disk access and block-cache perspective > in many big-data applications, but doing so is infeasible from a random seek > perspective. > The PrefixTrie block encoding format attempts to solve both of these > problems. Some features: > * trie format for row key encoding completely eliminates duplicate row keys > and encodes similar row keys into a standard trie structure which also saves > a lot of space > * the column family is currently stored once at the beginning of each block. > this could easily be modified to allow multiple family names per block > * all qualifiers in the block are stored in their own trie format which > caters nicely to wide rows. duplicate qualifers between rows are eliminated. > the size of this trie determines the width of the block's qualifier > fixed-width-int > * the minimum timestamp is stored at the beginning of the block, and deltas > are calculated from that. the maximum delta determines the width of the > block's timestamp fixed-width-int > The block is structured with metadata at the beginning, then a section for > the row trie, then the column trie, then the timestamp deltas, and then then > all the values. Most work is done in the row trie, where every leaf node > (corresponding to a row) contains a list of offsets/references corresponding > to the cells in that row. Each cell is fixed-width to enable binary > searching and is represented by [1 byte operationType, X bytes qualifier > offset, X bytes timestamp delta offset]. > If all operation types are the same for a block, there will be zero per-cell > overhead. Same for timestamps. Same for qualifiers when i get a chance. > So, the compression aspect is very strong, but makes a few small sacrifices > on VarInt size to enable faster binary searches in trie fan-out nodes. > A more compressed but slower version might build on this by also applying > further (suffix, etc) compression on the trie nodes at the cost of slower > write speed. Even further compression could be obtained by using all VInts > instead of FInts with a sacrifice on random seek speed (though not huge). > One current drawback is the current write speed. While programmed with good > constructs like TreeMaps, ByteBuffers, binary searches, etc, it's not > programmed with the same level of optimization as the read path. Work will > need to be done to optimize the data structures used for encoding and could > probably show a 10x increase. It will still be slower than delta encoding, > but with a much higher decode speed. I have not yet created a thor
[jira] [Commented] (HBASE-5993) Add a no-read Append
[ https://issues.apache.org/jira/browse/HBASE-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285800#comment-13285800 ] Jacques commented on HBASE-5993: The implementation of HBASE-4218, HBASE-4676 and HBASE-6093 reduce the storage overhead of the multicolumn approach to solving this problem. Network bandwidth and processing overhead will still exist. Using encoding schemes to solve this problem is nice because the changes are constrained as opposed to cross-cutting. That being said, it seems a bit like boiling the ocean to make a cup of tea. Let me put a design doc together and then we can reevaluate. My intuition is that this type of functionality could open up a new set of use cases for HBase. > Add a no-read Append > > > Key: HBASE-5993 > URL: https://issues.apache.org/jira/browse/HBASE-5993 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Jacques >Priority: Critical > > HBASE-4102 added an atomic append. For high performance situations, it would > be helpful to be able to do appends that don't actually require a read of the > existing value. This would be useful in building a growing set of values. > Our original use case was for implementing a form of search in HBase where a > cell would contain a list of document ids associated with a particular > keyword for search. However it seems like it would also be useful to provide > substantial performance improvements for most Append scenarios. > Within the client API, the simplest way to implement this would be to > leverage the existing Append api. If the Append is marked as > setReturnResults(false), use this code path. If result return is requested, > use the existing Append implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-4720: --- Attachment: 4720_trunk_v3.1.patch Fix the logging issue Andrew found. Thanks Stack, Andrew, and Ted for reviewing. Thanks Mubarak for the patch. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.96.0, 0.94.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, HBASE-4720.trunk.v1.patch, > HBASE-4720.trunk.v2.patch, HBASE-4720.trunk.v3.patch, > HBASE-4720.trunk.v4.patch, HBASE-4720.trunk.v5.patch, > HBASE-4720.trunk.v6.patch, HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, > HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285804#comment-13285804 ] David S. Wang commented on HBASE-6091: -- I have some content ready; however is there some format I should post it in (e.g. Confluence wiki, etc.)? > Come up with strawman proposal for RC testing matrix > > > Key: HBASE-6091 > URL: https://issues.apache.org/jira/browse/HBASE-6091 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.96.0 >Reporter: David S. Wang >Assignee: David S. Wang > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics
[ https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285808#comment-13285808 ] Hudson commented on HBASE-6131: --- Integrated in HBase-TRUNK #2953 (See [https://builds.apache.org/job/HBase-TRUNK/2953/]) HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 1344299) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java * /hbase/trunk/pom.xml > Add attribution for code added by HBASE-5533 metrics > > > Key: HBASE-6131 > URL: https://issues.apache.org/jira/browse/HBASE-6131 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.94.1 > > Attachments: 6131.txt, 6131_094.txt > > > See the comment over in > https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920 > The metrics histogram code was copied w/o attribution. Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5533) Add more metrics to HBase
[ https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285807#comment-13285807 ] Hudson commented on HBASE-5533: --- Integrated in HBase-TRUNK #2953 (See [https://builds.apache.org/job/HBase-TRUNK/2953/]) HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 1344299) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java * /hbase/trunk/pom.xml > Add more metrics to HBase > - > > Key: HBASE-5533 > URL: https://issues.apache.org/jira/browse/HBASE-5533 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.94.0 >Reporter: Shaneal Manek >Assignee: Shaneal Manek >Priority: Minor > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: BlockingQueueContention.java, HBASE-5533-0.92-v4.patch, > HBASE-5533-TRUNK-v6.patch, HBASE-5533-TRUNK-v6.patch, > HBASE-5533-v7-0.92.patch, TimingOverhead.java, hbase-5533-0.92.patch, > hbase5533-0.92-v2.patch, hbase5533-0.92-v3.patch, hbase5533-0.92-v5.patch, > histogram_web_ui.png > > > To debug/monitor production clusters, there are some more metrics I wish I > had available. > In particular: > - Although the average FS latencies are useful, a 'histogram' of recent > latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc) > would be more useful > - Similar histograms of latencies on common operations (GET, PUT, DELETE) > would be useful > - Counting the number of accesses to each region to detect hotspotting > - Exposing the current number of HLog files -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[
https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285809#comment-13285809
]
Zhihong Yu commented on HBASE-6109:
---
Rename TestLocker class to TestKeyLocker ?
{code}
+// It has no reason to be a lock shares with the other operations.
{code}
'shares with' -> 'shared with'
Indentation in AssignmentManager.addToRITandCallClose() was off. It would be
nice to correct the existing lines.
{code}
+// No lock concurrency: adding a share synchronized here would not prevent
to have two
+// entries as we don't check if the region is already there. This must be
ensured by the
+// method callers. todo nli: check
{code}
'share synchronized' -> 'synchronized'. Remove the 'todo nli:' at the end.
{code}
-Map plans=new HashMap();
+Map plans=new HashMap(regions.size());
{code}
Insert spaces around = sign.
{code}
+ * @return True if none of the regions in the set are in transition
{code}
'are in' -> 'is in'
{code}
+ public NavigableMap copyMap() {
+return delegatee.clone();
{code}
Why not call the method clone() ?
{code}
+ public void clear() {
+if (!delegatee.isEmpty()) {
+ synchronized (delegatee) {
{code}
Suppose delegatee is empty upon entry to the above method, what if an entry is
added after the isEmpty() check ?
> Improve RIT performances during assignment on large clusters
>
>
> Key: HBASE-6109
> URL: https://issues.apache.org/jira/browse/HBASE-6109
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 6109.v19.patch, 6109.v7.patch
>
>
> The main points in this patch are:
> - lowering the number of copy of the RIT list
> - lowering the number of synchronization
> - synchronizing on a region rather than on everything
> It also contains:
> - some fixes around the RIT notification: the list was sometimes modified
> without a corresponding 'notify'.
> - some tests flakiness correction, actually unrelated to this patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5533) Add more metrics to HBase
[ https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285813#comment-13285813 ] Hudson commented on HBASE-5533: --- Integrated in HBase-0.94 #231 (See [https://builds.apache.org/job/HBase-0.94/231/]) HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 1344305) HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 1344304) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java stack : Files : * /hbase/branches/0.94/pom.xml * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java > Add more metrics to HBase > - > > Key: HBASE-5533 > URL: https://issues.apache.org/jira/browse/HBASE-5533 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.94.0 >Reporter: Shaneal Manek >Assignee: Shaneal Manek >Priority: Minor > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: BlockingQueueContention.java, HBASE-5533-0.92-v4.patch, > HBASE-5533-TRUNK-v6.patch, HBASE-5533-TRUNK-v6.patch, > HBASE-5533-v7-0.92.patch, TimingOverhead.java, hbase-5533-0.92.patch, > hbase5533-0.92-v2.patch, hbase5533-0.92-v3.patch, hbase5533-0.92-v5.patch, > histogram_web_ui.png > > > To debug/monitor production clusters, there are some more metrics I wish I > had available. > In particular: > - Although the average FS latencies are useful, a 'histogram' of recent > latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc) > would be more useful > - Similar histograms of latencies on common operations (GET, PUT, DELETE) > would be useful > - Counting the number of accesses to each region to detect hotspotting > - Exposing the current number of HLog files -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6131) Add attribution for code added by HBASE-5533 metrics
[ https://issues.apache.org/jira/browse/HBASE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285814#comment-13285814 ] Hudson commented on HBASE-6131: --- Integrated in HBase-0.94 #231 (See [https://builds.apache.org/job/HBase-0.94/231/]) HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 1344305) HBASE-6131 Add attribution for code added by HBASE-5533 metrics (Revision 1344304) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java stack : Files : * /hbase/branches/0.94/pom.xml * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java > Add attribution for code added by HBASE-5533 metrics > > > Key: HBASE-6131 > URL: https://issues.apache.org/jira/browse/HBASE-6131 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.94.1 > > Attachments: 6131.txt, 6131_094.txt > > > See the comment over in > https://issues.apache.org/jira/browse/HBASE-5533?focusedCommentId=13283920&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13283920 > The metrics histogram code was copied w/o attribution. Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[
https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285821#comment-13285821
]
Zhihong Yu commented on HBASE-6109:
---
{code}
+ // A number of lock we want to easily support. It's not a maximum.
{code}
'A number' -> 'The number'
{code}
+ // We need an atomic counter to manage the number of people using the lock
and free it when
+ // it's equals to zero.
{code}
'number of people' -> 'number of users'
'it's equals to zero.' -> 'it's equal to zero.'
{code}
+ static class RegionLock extends ReentrantLock {
{code}
The outer class is generic. The inner class shouldn't mention Region.
> Improve RIT performances during assignment on large clusters
>
>
> Key: HBASE-6109
> URL: https://issues.apache.org/jira/browse/HBASE-6109
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 6109.v19.patch, 6109.v7.patch
>
>
> The main points in this patch are:
> - lowering the number of copy of the RIT list
> - lowering the number of synchronization
> - synchronizing on a region rather than on everything
> It also contains:
> - some fixes around the RIT notification: the list was sometimes modified
> without a corresponding 'notify'.
> - some tests flakiness correction, actually unrelated to this patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6049) Serializing "List" containing null elements will cause NullPointerException in HbaseObjectWritable.writeObject()
[ https://issues.apache.org/jira/browse/HBASE-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285822#comment-13285822 ] Hadoop QA commented on HBASE-6049: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530224/6049-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2054//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2054//console This message is automatically generated. > Serializing "List" containing null elements will cause NullPointerException > in HbaseObjectWritable.writeObject() > > > Key: HBASE-6049 > URL: https://issues.apache.org/jira/browse/HBASE-6049 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.94.0 >Reporter: Maryann Xue > Attachments: 6049-v3.patch, HBASE-6049-v2.patch, HBASE-6049-v3.patch, > HBASE-6049.patch > > > An error case could be in Coprocessor AggregationClient, the median() > function handles an empty region and returns a List Object with the first > element as a Null value. NPE occurs in the RPC response stage and the > response never gets sent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6122) Backup master does not become Active master after ZK exception
[
https://issues.apache.org/jira/browse/HBASE-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-6122:
--
Fix Version/s: (was: 0.96.0)
> Backup master does not become Active master after ZK exception
> --
>
> Key: HBASE-6122
> URL: https://issues.apache.org/jira/browse/HBASE-6122
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-6122_0.92.patch, HBASE-6122_0.94.patch
>
>
> -> Active master gets ZK expiry exception.
> -> Backup master becomes active.
> -> The previous active master retries and becomes the back up master.
> Now when the new active master goes down and the current back up master comes
> up, it goes down again with the zk expiry exception it got in the first step.
> {code}
> if (abortNow(msg, t)) {
> if (t != null) LOG.fatal(msg, t);
> else LOG.fatal(msg);
> this.abort = true;
> stop("Aborting");
> }
> {code}
> In ActiveMasterManager.blockUntilBecomingActiveMaster we try to wait till the
> back up master becomes active.
> {code}
> synchronized (this.clusterHasActiveMaster) {
> while (this.clusterHasActiveMaster.get() && !this.master.isStopped()) {
> try {
> this.clusterHasActiveMaster.wait();
> } catch (InterruptedException e) {
> // We expect to be interrupted when a master dies, will fall out if
> so
> LOG.debug("Interrupted waiting for master to die", e);
> }
> }
> if (!clusterStatusTracker.isClusterUp()) {
> this.master.stop("Cluster went down before this master became
> active");
> }
> if (this.master.isStopped()) {
> return cleanSetOfActiveMaster;
> }
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> }
> return cleanSetOfActiveMaster;
> {code}
> When the back up master (it is in back up mode as he got ZK exception), once
> again tries to come to active we don't get the return value that comes out
> from
> {code}
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> {code}
> We tend to return the 'cleanSetOfActiveMaster' which was previously false.
> Now because of this instead of again becoming active the back up master goes
> down in the abort() code. Thanks to Gopi,my colleague for reporting this
> issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6049) Serializing "List" containing null elements will cause NullPointerException in HbaseObjectWritable.writeObject()
[ https://issues.apache.org/jira/browse/HBASE-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-6049: -- Fix Version/s: 0.94.1 0.96.0 Hadoop Flags: Reviewed @Stack: Do you think patch v3 can be integrated ? > Serializing "List" containing null elements will cause NullPointerException > in HbaseObjectWritable.writeObject() > > > Key: HBASE-6049 > URL: https://issues.apache.org/jira/browse/HBASE-6049 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.94.0 >Reporter: Maryann Xue > Fix For: 0.96.0, 0.94.1 > > Attachments: 6049-v3.patch, HBASE-6049-v2.patch, HBASE-6049-v3.patch, > HBASE-6049.patch > > > An error case could be in Coprocessor AggregationClient, the median() > function handles an empty region and returns a List Object with the first > element as a Null value. NPE occurs in the RPC response stage and the > response never gets sent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6122) Backup master does not become Active master after ZK exception
[
https://issues.apache.org/jira/browse/HBASE-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285823#comment-13285823
]
ramkrishna.s.vasudevan commented on HBASE-6122:
---
Committed to 0.92 and 0.94.
Thanks for the review Lars.
> Backup master does not become Active master after ZK exception
> --
>
> Key: HBASE-6122
> URL: https://issues.apache.org/jira/browse/HBASE-6122
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-6122_0.92.patch, HBASE-6122_0.94.patch
>
>
> -> Active master gets ZK expiry exception.
> -> Backup master becomes active.
> -> The previous active master retries and becomes the back up master.
> Now when the new active master goes down and the current back up master comes
> up, it goes down again with the zk expiry exception it got in the first step.
> {code}
> if (abortNow(msg, t)) {
> if (t != null) LOG.fatal(msg, t);
> else LOG.fatal(msg);
> this.abort = true;
> stop("Aborting");
> }
> {code}
> In ActiveMasterManager.blockUntilBecomingActiveMaster we try to wait till the
> back up master becomes active.
> {code}
> synchronized (this.clusterHasActiveMaster) {
> while (this.clusterHasActiveMaster.get() && !this.master.isStopped()) {
> try {
> this.clusterHasActiveMaster.wait();
> } catch (InterruptedException e) {
> // We expect to be interrupted when a master dies, will fall out if
> so
> LOG.debug("Interrupted waiting for master to die", e);
> }
> }
> if (!clusterStatusTracker.isClusterUp()) {
> this.master.stop("Cluster went down before this master became
> active");
> }
> if (this.master.isStopped()) {
> return cleanSetOfActiveMaster;
> }
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> }
> return cleanSetOfActiveMaster;
> {code}
> When the back up master (it is in back up mode as he got ZK exception), once
> again tries to come to active we don't get the return value that comes out
> from
> {code}
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> {code}
> We tend to return the 'cleanSetOfActiveMaster' which was previously false.
> Now because of this instead of again becoming active the back up master goes
> down in the abort() code. Thanks to Gopi,my colleague for reporting this
> issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6122) Backup master does not become Active master after ZK exception
[
https://issues.apache.org/jira/browse/HBASE-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan resolved HBASE-6122.
---
Resolution: Fixed
Hadoop Flags: Reviewed
> Backup master does not become Active master after ZK exception
> --
>
> Key: HBASE-6122
> URL: https://issues.apache.org/jira/browse/HBASE-6122
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-6122_0.92.patch, HBASE-6122_0.94.patch
>
>
> -> Active master gets ZK expiry exception.
> -> Backup master becomes active.
> -> The previous active master retries and becomes the back up master.
> Now when the new active master goes down and the current back up master comes
> up, it goes down again with the zk expiry exception it got in the first step.
> {code}
> if (abortNow(msg, t)) {
> if (t != null) LOG.fatal(msg, t);
> else LOG.fatal(msg);
> this.abort = true;
> stop("Aborting");
> }
> {code}
> In ActiveMasterManager.blockUntilBecomingActiveMaster we try to wait till the
> back up master becomes active.
> {code}
> synchronized (this.clusterHasActiveMaster) {
> while (this.clusterHasActiveMaster.get() && !this.master.isStopped()) {
> try {
> this.clusterHasActiveMaster.wait();
> } catch (InterruptedException e) {
> // We expect to be interrupted when a master dies, will fall out if
> so
> LOG.debug("Interrupted waiting for master to die", e);
> }
> }
> if (!clusterStatusTracker.isClusterUp()) {
> this.master.stop("Cluster went down before this master became
> active");
> }
> if (this.master.isStopped()) {
> return cleanSetOfActiveMaster;
> }
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> }
> return cleanSetOfActiveMaster;
> {code}
> When the back up master (it is in back up mode as he got ZK exception), once
> again tries to come to active we don't get the return value that comes out
> from
> {code}
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> {code}
> We tend to return the 'cleanSetOfActiveMaster' which was previously false.
> Now because of this instead of again becoming active the back up master goes
> down in the abort() code. Thanks to Gopi,my colleague for reporting this
> issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5722) NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW down.
[
https://issues.apache.org/jira/browse/HBASE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan resolved HBASE-5722.
---
Resolution: Fixed
> NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW
> down.
> --
>
> Key: HBASE-5722
> URL: https://issues.apache.org/jira/browse/HBASE-5722
> Project: HBase
> Issue Type: Bug
> Components: zookeeper
>Affects Versions: 0.94.0, 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5722.patch
>
>
> {code}
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> for (String node: nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte [] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> {code}
> The above code can throw NPE when listChildrenAndWatchForNewChildren returns
> null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5722) NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW down.
[
https://issues.apache.org/jira/browse/HBASE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285825#comment-13285825
]
ramkrishna.s.vasudevan commented on HBASE-5722:
---
The comment above mentioned that the commit was done to 0.94, but it wasn't. I
have committed it to 0.94. Just a mere copy paste. ensured that the test case
passes.
> NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW
> down.
> --
>
> Key: HBASE-5722
> URL: https://issues.apache.org/jira/browse/HBASE-5722
> Project: HBase
> Issue Type: Bug
> Components: zookeeper
>Affects Versions: 0.94.0, 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5722.patch
>
>
> {code}
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> for (String node: nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte [] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> {code}
> The above code can throw NPE when listChildrenAndWatchForNewChildren returns
> null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-5733. --- Resolution: Fixed Committed to 0.94 and 0.92. Hence resolving it. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6109: --- Status: Open (was: Patch Available) > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285828#comment-13285828 ] nkeywal commented on HBASE-6109: bq. Rename TestLocker class to TestKeyLocker ? Done. bq. 'shares with' -> 'shared with' Done. bq. Indentation in AssignmentManager.addToRITandCallClose() was off. It would be nice to correct the existing lines. Done bq. 'share synchronized' -> 'synchronized'. Remove the 'todo nli:' at the end. Done bq. Insert spaces around = sign. Done. bq. 'are in' -> 'is in' Done bq. Why not call the method clone() ? We don't really want the NotifiableConcurrentSkipListMap to be cloneable: however, some functions want to work on a copy of the data structure, for reporting or test (with all the 'Map' semantic), hence the internal clone. bq. Suppose delegatee is empty upon entry to the above method, what if an entry is added after the isEmpty() check ? It will be equivalent to adding it just after the clear. bq. 'A number' -> 'The number' Done. bq. 'number of people' -> 'number of users' Done bq. 'it's equals to zero.' -> 'it's equal to zero.' Done bq. The outer class is generic. The inner class shouldn't mention Region. Done > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6109: --- Attachment: 6109.v21.patch > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v21.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4676) Prefix Compression - Trie data block encoding
[ https://issues.apache.org/jira/browse/HBASE-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285830#comment-13285830 ] Matt Corgan commented on HBASE-4676: Jacques - I like the idea , and I think the long-term goal is to auto-tune some of this stuff. For example, if flushes or compactions are backlogged, that maybe a good signal for certain tables to temporarily switch from more compact Trie/GZip encoding/compression to faster-encoding Prefix/Snappy. There are other factors at play though on a table-by-table basis. Some tables might be random-read heavy, in which case you would want to keep the Trie. > Prefix Compression - Trie data block encoding > - > > Key: HBASE-4676 > URL: https://issues.apache.org/jira/browse/HBASE-4676 > Project: HBase > Issue Type: New Feature > Components: io, performance, regionserver >Affects Versions: 0.90.6 >Reporter: Matt Corgan >Assignee: Matt Corgan > Attachments: HBASE-4676-0.94-v1.patch, PrefixTrie_Format_v1.pdf, > PrefixTrie_Performance_v1.pdf, SeeksPerSec by blockSize.png, > hbase-prefix-trie-0.1.jar > > > The HBase data block format has room for 2 significant improvements for > applications that have high block cache hit ratios. > First, there is no prefix compression, and the current KeyValue format is > somewhat metadata heavy, so there can be tremendous memory bloat for many > common data layouts, specifically those with long keys and short values. > Second, there is no random access to KeyValues inside data blocks. This > means that every time you double the datablock size, average seek time (or > average cpu consumption) goes up by a factor of 2. The standard 64KB block > size is ~10x slower for random seeks than a 4KB block size, but block sizes > as small as 4KB cause problems elsewhere. Using block sizes of 256KB or 1MB > or more may be more efficient from a disk access and block-cache perspective > in many big-data applications, but doing so is infeasible from a random seek > perspective. > The PrefixTrie block encoding format attempts to solve both of these > problems. Some features: > * trie format for row key encoding completely eliminates duplicate row keys > and encodes similar row keys into a standard trie structure which also saves > a lot of space > * the column family is currently stored once at the beginning of each block. > this could easily be modified to allow multiple family names per block > * all qualifiers in the block are stored in their own trie format which > caters nicely to wide rows. duplicate qualifers between rows are eliminated. > the size of this trie determines the width of the block's qualifier > fixed-width-int > * the minimum timestamp is stored at the beginning of the block, and deltas > are calculated from that. the maximum delta determines the width of the > block's timestamp fixed-width-int > The block is structured with metadata at the beginning, then a section for > the row trie, then the column trie, then the timestamp deltas, and then then > all the values. Most work is done in the row trie, where every leaf node > (corresponding to a row) contains a list of offsets/references corresponding > to the cells in that row. Each cell is fixed-width to enable binary > searching and is represented by [1 byte operationType, X bytes qualifier > offset, X bytes timestamp delta offset]. > If all operation types are the same for a block, there will be zero per-cell > overhead. Same for timestamps. Same for qualifiers when i get a chance. > So, the compression aspect is very strong, but makes a few small sacrifices > on VarInt size to enable faster binary searches in trie fan-out nodes. > A more compressed but slower version might build on this by also applying > further (suffix, etc) compression on the trie nodes at the cost of slower > write speed. Even further compression could be obtained by using all VInts > instead of FInts with a sacrifice on random seek speed (though not huge). > One current drawback is the current write speed. While programmed with good > constructs like TreeMaps, ByteBuffers, binary searches, etc, it's not > programmed with the same level of optimization as the read path. Work will > need to be done to optimize the data structures used for encoding and could > probably show a 10x increase. It will still be slower than delta encoding, > but with a much higher decode speed. I have not yet created a thorough > benchmark for write speed nor sequential read speed. > Though the trie is reaching a point where it is internally very efficient > (probably within half or a quarter of its max read speed) the way that hbase > currently uses it is far from optimal. The KeyValueScanner and related > classe
[jira] [Commented] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285831#comment-13285831 ] Hadoop QA commented on HBASE-4720: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530230/4720_trunk_v3.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager org.apache.hadoop.hbase.master.TestLogsCleaner org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2055//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2055//console This message is automatically generated. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.96.0, 0.94.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, HBASE-4720.trunk.v1.patch, > HBASE-4720.trunk.v2.patch, HBASE-4720.trunk.v3.patch, > HBASE-4720.trunk.v4.patch, HBASE-4720.trunk.v5.patch, > HBASE-4720.trunk.v6.patch, HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, > HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4720: - Status: Open (was: Patch Available) > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.96.0, 0.94.1, 0.92.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, HBASE-4720.trunk.v1.patch, > HBASE-4720.trunk.v2.patch, HBASE-4720.trunk.v3.patch, > HBASE-4720.trunk.v4.patch, HBASE-4720.trunk.v5.patch, > HBASE-4720.trunk.v6.patch, HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, > HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285840#comment-13285840 ] Zhihong Yu commented on HBASE-6109: --- @N: Thanks for the quick turn-around. > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v21.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4720: - Attachment: 4720_trunk_v3.patch Retry just in case > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.96.0, 0.94.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, 4720_trunk_v3.patch, > HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4720: - Status: Patch Available (was: Open) > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.96.0, 0.94.1, 0.92.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, 4720_trunk_v3.patch, > HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6109: - Status: Patch Available (was: Open) > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v21.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6049) Serializing "List" containing null elements will cause NullPointerException in HbaseObjectWritable.writeObject()
[ https://issues.apache.org/jira/browse/HBASE-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6049: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to 0.94 and trunk. Thanks for the patch Maryann and thanks for rebasing Ted. > Serializing "List" containing null elements will cause NullPointerException > in HbaseObjectWritable.writeObject() > > > Key: HBASE-6049 > URL: https://issues.apache.org/jira/browse/HBASE-6049 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.94.0 >Reporter: Maryann Xue > Fix For: 0.96.0, 0.94.1 > > Attachments: 6049-v3.patch, HBASE-6049-v2.patch, HBASE-6049-v3.patch, > HBASE-6049.patch > > > An error case could be in Coprocessor AggregationClient, the median() > function handles an empty region and returns a List Object with the first > element as a Null value. NPE occurs in the RPC response stage and the > response never gets sent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.
[ https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HBASE-5892: --- Attachment: hbase-5892-3.patch > [hbck] Refactor parallel WorkItem* to Futures. > -- > > Key: HBASE-5892 > URL: https://issues.apache.org/jira/browse/HBASE-5892 > Project: HBase > Issue Type: Improvement >Reporter: Jonathan Hsieh >Assignee: Andrew Wang > Labels: noob > Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, > hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892.patch > > > This would convert WorkItem* logic (with low level notifies, and rough > exception handling) into a more canonical Futures pattern. > Currently there are two instances of this pattern (for loading hdfs dirs, for > contacting regionservers for assignments, and soon -- for loading hdfs > .regioninfo files). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6049) Serializing "List" containing null elements will cause NullPointerException in HbaseObjectWritable.writeObject()
[ https://issues.apache.org/jira/browse/HBASE-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285852#comment-13285852 ] stack commented on HBASE-6049: -- @Maryann I am having trouble adding you as a committer. Will keep trying. > Serializing "List" containing null elements will cause NullPointerException > in HbaseObjectWritable.writeObject() > > > Key: HBASE-6049 > URL: https://issues.apache.org/jira/browse/HBASE-6049 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.94.0 >Reporter: Maryann Xue > Fix For: 0.96.0, 0.94.1 > > Attachments: 6049-v3.patch, HBASE-6049-v2.patch, HBASE-6049-v3.patch, > HBASE-6049.patch > > > An error case could be in Coprocessor AggregationClient, the median() > function handles an empty region and returns a List Object with the first > element as a Null value. NPE occurs in the RPC response stage and the > response never gets sent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.
[ https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285854#comment-13285854 ] Andrew Wang commented on HBASE-5892: Re-diffed to get the right prefix. Thanks Zhihong. > [hbck] Refactor parallel WorkItem* to Futures. > -- > > Key: HBASE-5892 > URL: https://issues.apache.org/jira/browse/HBASE-5892 > Project: HBase > Issue Type: Improvement >Reporter: Jonathan Hsieh >Assignee: Andrew Wang > Labels: noob > Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch, > hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892.patch > > > This would convert WorkItem* logic (with low level notifies, and rough > exception handling) into a more canonical Futures pattern. > Currently there are two instances of this pattern (for loading hdfs dirs, for > contacting regionservers for assignments, and soon -- for loading hdfs > .regioninfo files). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5609) Add the ability to pass additional information for slow query logging
[ https://issues.apache.org/jira/browse/HBASE-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Drzal updated HBASE-5609: - Attachment: HBASE-5609.patch First shot at this > Add the ability to pass additional information for slow query logging > - > > Key: HBASE-5609 > URL: https://issues.apache.org/jira/browse/HBASE-5609 > Project: HBase > Issue Type: New Feature > Components: client, ipc >Reporter: Michael Drzal >Assignee: Michael Drzal >Priority: Minor > Attachments: HBASE-5609.patch > > > HBase-4117 added the ability to log information about queries that returned > too much data or ran for too long. There is some information written as a > fingerprint that can be used to tell what table/column families/... are > affected. I would like to extend this functionality to allow the client to > insert an identifier into the operation that gets output in the log. The > idea behind this would be that if there were N places in the client > application that touched a given table in a certain way, you could quickly > narrow things down by inserting a className:functionName or similar > identifier. I'm fully willing to go back on this if people think that it > isn't a problem in real life and it would just add complexity to the code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6089) SSH and AM.joinCluster causes Concurrent Modification exception.
[ https://issues.apache.org/jira/browse/HBASE-6089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285863#comment-13285863 ] stack commented on HBASE-6089: -- bq. What you feel? If it is ok for you , i can commit this? Yes. Its no good adding a CSLM as I suggested because the sync block is needed to cover regions and servers updates (as Anoop says). > SSH and AM.joinCluster causes Concurrent Modification exception. > > > Key: HBASE-6089 > URL: https://issues.apache.org/jira/browse/HBASE-6089 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1, 0.94.0 >Reporter: ramkrishna.s.vasudevan >Assignee: rajeshbabu > Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-6089_92.patch, HBASE-6089_94.patch, > HBASE-6089_trunk.patch > > > AM.regions map is parallely accessed in SSH and Master initialization leading > to ConcurrentModificationException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5892) [hbck] Refactor parallel WorkItem* to Futures.
[
https://issues.apache.org/jira/browse/HBASE-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285864#comment-13285864
]
Zhihong Yu commented on HBASE-5892:
---
Minor comment:
{code}
+List work = new
ArrayList(regionServerList.size());
{code}
variable work should be named works.
{code}
+if(hdfsEntry.hri != null) {
{code}
Insert space between if and (.
> [hbck] Refactor parallel WorkItem* to Futures.
> --
>
> Key: HBASE-5892
> URL: https://issues.apache.org/jira/browse/HBASE-5892
> Project: HBase
> Issue Type: Improvement
>Reporter: Jonathan Hsieh
>Assignee: Andrew Wang
> Labels: noob
> Attachments: hbase-5892-1.patch, hbase-5892-2-0.90.patch,
> hbase-5892-2.patch, hbase-5892-3.patch, hbase-5892.patch
>
>
> This would convert WorkItem* logic (with low level notifies, and rough
> exception handling) into a more canonical Futures pattern.
> Currently there are two instances of this pattern (for loading hdfs dirs, for
> contacting regionservers for assignments, and soon -- for loading hdfs
> .regioninfo files).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6049) Serializing "List" containing null elements will cause NullPointerException in HbaseObjectWritable.writeObject()
[ https://issues.apache.org/jira/browse/HBASE-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-6049: - Assignee: Maryann Xue > Serializing "List" containing null elements will cause NullPointerException > in HbaseObjectWritable.writeObject() > > > Key: HBASE-6049 > URL: https://issues.apache.org/jira/browse/HBASE-6049 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.94.0 >Reporter: Maryann Xue >Assignee: Maryann Xue > Fix For: 0.96.0, 0.94.1 > > Attachments: 6049-v3.patch, HBASE-6049-v2.patch, HBASE-6049-v3.patch, > HBASE-6049.patch > > > An error case could be in Coprocessor AggregationClient, the median() > function handles an empty region and returns a List Object with the first > element as a Null value. NPE occurs in the RPC response stage and the > response never gets sent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6122) Backup master does not become Active master after ZK exception
[
https://issues.apache.org/jira/browse/HBASE-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285871#comment-13285871
]
Hudson commented on HBASE-6122:
---
Integrated in HBase-0.94 #233 (See
[https://builds.apache.org/job/HBase-0.94/233/])
HBASE-6122 Backup master does not become Active master after ZK exception
(Ram) (Revision 1344348)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
> Backup master does not become Active master after ZK exception
> --
>
> Key: HBASE-6122
> URL: https://issues.apache.org/jira/browse/HBASE-6122
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-6122_0.92.patch, HBASE-6122_0.94.patch
>
>
> -> Active master gets ZK expiry exception.
> -> Backup master becomes active.
> -> The previous active master retries and becomes the back up master.
> Now when the new active master goes down and the current back up master comes
> up, it goes down again with the zk expiry exception it got in the first step.
> {code}
> if (abortNow(msg, t)) {
> if (t != null) LOG.fatal(msg, t);
> else LOG.fatal(msg);
> this.abort = true;
> stop("Aborting");
> }
> {code}
> In ActiveMasterManager.blockUntilBecomingActiveMaster we try to wait till the
> back up master becomes active.
> {code}
> synchronized (this.clusterHasActiveMaster) {
> while (this.clusterHasActiveMaster.get() && !this.master.isStopped()) {
> try {
> this.clusterHasActiveMaster.wait();
> } catch (InterruptedException e) {
> // We expect to be interrupted when a master dies, will fall out if
> so
> LOG.debug("Interrupted waiting for master to die", e);
> }
> }
> if (!clusterStatusTracker.isClusterUp()) {
> this.master.stop("Cluster went down before this master became
> active");
> }
> if (this.master.isStopped()) {
> return cleanSetOfActiveMaster;
> }
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> }
> return cleanSetOfActiveMaster;
> {code}
> When the back up master (it is in back up mode as he got ZK exception), once
> again tries to come to active we don't get the return value that comes out
> from
> {code}
> // Try to become active master again now that there is no active master
> blockUntilBecomingActiveMaster(startupStatus,clusterStatusTracker);
> {code}
> We tend to return the 'cleanSetOfActiveMaster' which was previously false.
> Now because of this instead of again becoming active the back up master goes
> down in the abort() code. Thanks to Gopi,my colleague for reporting this
> issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285873#comment-13285873 ] Hudson commented on HBASE-5733: --- Integrated in HBase-0.94 #233 (See [https://builds.apache.org/job/HBase-0.94/233/]) HBASE-5733 AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. (Uma) (Revision 1344352) Result = FAILURE ramkrishna : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5722) NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW down.
[
https://issues.apache.org/jira/browse/HBASE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285872#comment-13285872
]
Hudson commented on HBASE-5722:
---
Integrated in HBase-0.94 #233 (See
[https://builds.apache.org/job/HBase-0.94/233/])
HBASE-5722 NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not
available or NW down. (Uma) (Revision 1344351)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
> NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW
> down.
> --
>
> Key: HBASE-5722
> URL: https://issues.apache.org/jira/browse/HBASE-5722
> Project: HBase
> Issue Type: Bug
> Components: zookeeper
>Affects Versions: 0.94.0, 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5722.patch
>
>
> {code}
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> for (String node: nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte [] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> {code}
> The above code can throw NPE when listChildrenAndWatchForNewChildren returns
> null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285876#comment-13285876 ] Hadoop QA commented on HBASE-6109: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530237/6109.v21.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 26 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestDrainingServer org.apache.hadoop.hbase.master.TestAssignmentManager org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2056//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2056//console This message is automatically generated. > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v21.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285877#comment-13285877 ] Hadoop QA commented on HBASE-4720: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530241/4720_trunk_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2057//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2057//console This message is automatically generated. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.96.0, 0.94.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, 4720_trunk_v3.patch, > HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
Elliott Clark created HBASE-6135: Summary: Style the Web UI to use Twitter's Bootstrap. Key: HBASE-6135 URL: https://issues.apache.org/jira/browse/HBASE-6135 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Fix For: 0.96.0 Our web ui has lagged a little bit behind. While it's not a huge deal, it is one of the first things that new people see. As such styling it a little bit better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6134) Improvement for split-worker to speed up distributed-split-log
[
https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285891#comment-13285891
]
Zhihong Yu commented on HBASE-6134:
---
Test failed with NPE at:
{code}
for (FileStatus fileStatus : listStatus1) {
{code}
Checking for null wouldn't help either:
{code}
Failed tests:
testSequentialEditLogSeqNum(org.apache.hadoop.hbase.regionserver.wal.TestWALReplay):
The sequence number of the recoverd.edits and the current edit seq should be
same expected:<17> but was:<0>
{code}
> Improvement for split-worker to speed up distributed-split-log
> --
>
> Key: HBASE-6134
> URL: https://issues.apache.org/jira/browse/HBASE-6134
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6134.patch
>
>
> First，we do the test between local-master-split and distributed split log
> Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths
> splitting work)
> local-master-split:60s+
> distributed-split-log:165s+
> In fact, in our production environment, distributed-split-log also took 60s
> with 30 regionservers for 34 hlog files (regionserver may be in high load)
> We found split-worker split one log file took about 20s.
> I think we should do the improvement for this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4720: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.92, 0.94 and trunk. Thanks for the patch Jimmy and Mubarak. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.96.0, 0.94.1, 0.92.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, 4720_trunk_v3.patch, > HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285907#comment-13285907 ] Mubarak Seyed commented on HBASE-4720: -- Thanks Jimmy for the patch. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.96.0, 0.94.1 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, 4720_trunk_v3.patch, > HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6109) Improve RIT performances during assignment on large clusters
[ https://issues.apache.org/jira/browse/HBASE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285909#comment-13285909 ] nkeywal commented on HBASE-6109: I need to have a look at this one. org.apache.hadoop.hbase.master.TestAssignmentManager.testRegionInOpeningStateOnDeadRSWhileMasterFailover > Improve RIT performances during assignment on large clusters > > > Key: HBASE-6109 > URL: https://issues.apache.org/jira/browse/HBASE-6109 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 6109.v19.patch, 6109.v21.patch, 6109.v7.patch > > > The main points in this patch are: > - lowering the number of copy of the RIT list > - lowering the number of synchronization > - synchronizing on a region rather than on everything > It also contains: > - some fixes around the RIT notification: the list was sometimes modified > without a corresponding 'notify'. > - some tests flakiness correction, actually unrelated to this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
