[jira] [Created] (HBASE-5834) CopyTable usage is incorrect
CopyTable usage is incorrect Key: HBASE-5834 URL: https://issues.apache.org/jira/browse/HBASE-5834 Project: HBase Issue Type: Bug Reporter: Zhihong Yu The example given here is outdated: http://hbase.apache.org/book/ops_mgt.html#copytable The classes for rs.class and rs.impl don't exist. Example in Java code needs to be updated as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5826) Improve sync of HLog edits
Improve sync of HLog edits -- Key: HBASE-5826 URL: https://issues.apache.org/jira/browse/HBASE-5826 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu HBASE-5782 solved the correctness issue for the sync of HLog edits. Todd provided a patch that would achieve higher throughput. This JIRA is a continuation of Todd's work submitted there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5769) Use builder pattern to create HServerLoad.RegionLoad
Use builder pattern to create HServerLoad.RegionLoad Key: HBASE-5769 URL: https://issues.apache.org/jira/browse/HBASE-5769 Project: HBase Issue Type: Task Reporter: Zhihong Yu Currently, HRegionServer.createRegionLoad() calls RegionLoad ctor with all the parameters. This makes adding new members to RegionLoad tedious. Builder pattern should be employed to create HServerLoad.RegionLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5554) hadoop.native.lib config is deprecated
hadoop.native.lib config is deprecated
Key: HBASE-5554
URL: https://issues.apache.org/jira/browse/HBASE-5554
Project: HBase
Issue Type: Task
Reporter: Zhihong Yu
When using HBase shell, we see:
{code}
12/03/09 09:06:58 WARN conf.Configuration: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
{code}
io.native.lib.available should be used.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5512) Add support for INCLUDE_AND_SEEK_USING_HINT
Add support for INCLUDE_AND_SEEK_USING_HINT --- Key: HBASE-5512 URL: https://issues.apache.org/jira/browse/HBASE-5512 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu This came up from HBASE-2038 From Anoop: - What we wanted from the filter is include a row and then seek to the next row which we are interested in. I cant see such a facility with our Filter right now. Correct me if I am wrong. So suppose we already seeked to one row and this need to be included in the result, then the Filter should return INCLUDE. Then when the next next() call happens, then only we can return a SEEK_USING_HINT. So one extra row reading is needed. This might create even one unwanted HFileBlock fetch (who knows). Can we add reseek() at higher level? From Lars: Yep, for that we'd need to add INCLUDE_AND_SEEK_USING_HINT (similar to the INCLUDE_AND_SEEK_NEXT_ROW that we already have). Shouldn't be hard to add, I'm happy to do that, if that's the route we want to go with this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5478) TestMetaReaderEditor intermittently hangs in 0.92 branch
TestMetaReaderEditor intermittently hangs in 0.92 branch
Key: HBASE-5478
URL: https://issues.apache.org/jira/browse/HBASE-5478
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Zhihong Yu
From 0.92 build #304:
{code}
Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
Running org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 149.104 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5471) Upgrade hadoop to 1.0.2
Upgrade hadoop to 1.0.2 --- Key: HBASE-5471 URL: https://issues.apache.org/jira/browse/HBASE-5471 Project: HBase Issue Type: Task Reporter: Zhihong Yu Now that MAPREDUCE-3583 has been integrated, we should be prepared to upgrade hadoop to 1.0.2 Unit tests which fail on Apache QA machines due to NumberFormatException should pass after upgrade. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5456) Introduce PowerMock into our unit tests to reduce unnecessary method exposure
Introduce PowerMock into our unit tests to reduce unnecessary method exposure - Key: HBASE-5456 URL: https://issues.apache.org/jira/browse/HBASE-5456 Project: HBase Issue Type: Task Reporter: Zhihong Yu We should introduce PowerMock into our unit tests so that we don't have to expose methods intended to be used by unit tests. Here was Benoit's reply to a user of asynchbase about testability: OpenTSDB has unit tests that are mocking out HBaseClient just fine [1]. You can mock out pretty much anything on the JVM: final, private, JDK stuff, etc. All you need is the right tools. I've been very happy with PowerMock. It supports Mockito and EasyMock. I've never been keen on mutilating public interfaces for the sake of testing. With tools like PowerMock, we can keep the public APIs tidy while mocking and overriding anything, even in the most private guts of the classes. [1] https://github.com/stumbleupon/opentsdb/blob/master/src/uid/TestUniqueId.java#L66 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5435) TestForceCacheImportantBlocks fails with OutOfMemoryError
TestForceCacheImportantBlocks fails with OutOfMemoryError
-
Key: HBASE-5435
URL: https://issues.apache.org/jira/browse/HBASE-5435
Project: HBase
Issue Type: Test
Reporter: Zhihong Yu
Fix For: 0.94.0
Here is related stack trace (see
https://builds.apache.org/job/HBase-TRUNK/2665/testReport/org.apache.hadoop.hbase.io.hfile/TestForceCacheImportantBlocks/testCacheBlocks_1_/):
{code}
Caused by: java.lang.OutOfMemoryError
at java.util.zip.Deflater.init(Native Method)
at java.util.zip.Deflater.init(Deflater.java:124)
at java.util.zip.GZIPOutputStream.init(GZIPOutputStream.java:46)
at java.util.zip.GZIPOutputStream.init(GZIPOutputStream.java:58)
at
org.apache.hadoop.hbase.io.hfile.ReusableStreamGzipCodec$ReusableGzipOutputStream$ResetableGZIPOutputStream.init(ReusableStreamGzipCodec.java:79)
at
org.apache.hadoop.hbase.io.hfile.ReusableStreamGzipCodec$ReusableGzipOutputStream.init(ReusableStreamGzipCodec.java:90)
at
org.apache.hadoop.hbase.io.hfile.ReusableStreamGzipCodec.createOutputStream(ReusableStreamGzipCodec.java:130)
at
org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:101)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.createPlainCompressionStream(Compression.java:239)
at
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.createCompressionStream(Compression.java:223)
at
org.apache.hadoop.hbase.io.hfile.HFileWriterV1.getCompressingStream(HFileWriterV1.java:270)
at
org.apache.hadoop.hbase.io.hfile.HFileWriterV1.close(HFileWriterV1.java:416)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Writer.close(StoreFile.java:1115)
at
org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:706)
at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:633)
at org.apache.hadoop.hbase.regionserver.Store.access$400(Store.java:106)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5394) Add ability to include Protobufs in HbaseObjectWritable
Add ability to include Protobufs in HbaseObjectWritable --- Key: HBASE-5394 URL: https://issues.apache.org/jira/browse/HBASE-5394 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu This is a port of HADOOP-7379 This is to add the cases to HbaseObjectWritable to handle subclasses of Message, the superclass of codegenned protobufs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5366) Improving bulk table disable/enable/drop in shell
Improving bulk table disable/enable/drop in shell
-
Key: HBASE-5366
URL: https://issues.apache.org/jira/browse/HBASE-5366
Project: HBase
Issue Type: Task
Reporter: Zhihong Yu
HBASE-3506 added regex support for disabling, enabling and dropping tables.
Currently the list of tables are shown one on each line, this may lead to too
many lines:
{code}
tychangTable998
tychangTable999
Enable the above 901 tables (y/n)?
y
{code}
When the number of tables is high, each line should display multiple tables.
Disabling tables may take a long time. We should show the tables that have been
disabled in batches so that user knows the operation didn't hang.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5283) Request counters may become negative for heavily loaded regions
Request counters may become negative for heavily loaded regions
---
Key: HBASE-5283
URL: https://issues.apache.org/jira/browse/HBASE-5283
Project: HBase
Issue Type: Bug
Reporter: Zhihong Yu
Requests counter showing negative count, example under 'Requests' column:
-645470239
{code}
NameRegion Server Start Key End Key Requests
usertable,user2037516127892189021,1326756873774.16833e4566d1daef109b8fdcd1f4b5a6.
xxx.com:60030 user2037516127892189021 user2296868939942738705
-645470239
{code}
RegionLoad.readRequestsCount and RegionLoad.writeRequestsCount are of int type.
Our Ops has been running lots of heavy load operation.
RegionLoad.getRequestsCount() overflows int.MAX_VALUE. It is set to D986E7E1.
In table.jsp, RegionLoad.getRequestsCount() is assigned to long type. D986E7E1
is converted to long D986E7E1 which is -645470239 in decimal.
Suggested fix is to make readRequestsCount and writeRequestsCount long type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5270) Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler
Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler - Key: HBASE-5270 URL: https://issues.apache.org/jira/browse/HBASE-5270 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu Fix For: 0.94.0, 0.92.1 This JIRA continues the effort from HBASE-5179. Starting with Stack's comments about patches for 0.92 and TRUNK: Reviewing 0.92v17 isDeadServerInProgress is a new public method in ServerManager but it does not seem to be used anywhere. Does isDeadRootServerInProgress need to be public? Ditto for meta version. This method param names are not right 'definitiveRootServer'; what is meant by definitive? Do they need this qualifier? Is there anything in place to stop us expiring a server twice if its carrying root and meta? What is difference between asking assignment manager isCarryingRoot and this variable that is passed in? Should be doc'd at least. Ditto for meta. I think I've asked for this a few times - onlineServers needs to be explained... either in javadoc or in comment. This is the param passed into joinCluster. How does it arise? I think I know but am unsure. God love the poor noob that comes awandering this code trying to make sense of it all. It looks like we get the list by trawling zk for regionserver znodes that have not checked in. Don't we do this operation earlier in master setup? Are we doing it again here? Though distributed split log is configured, we will do in master single process splitting under some conditions with this patch. Its not explained in code why we would do this. Why do we think master log splitting 'high priority' when it could very well be slower. Should we only go this route if distributed splitting is not going on. Do we know if concurrent distributed log splitting and master splitting works? Why would we have dead servers in progress here in master startup? Because a servershutdownhandler fired? This patch is different to the patch for 0.90. Should go into trunk first with tests, then 0.92. Should it be in this issue? This issue is really hard to follow now. Maybe this issue is for 0.90.x and new issue for more work on this trunk patch? This patch needs to have the v18 differences applied. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5272) Simplify the logic in DefaultLoadBalancer
Simplify the logic in DefaultLoadBalancer - Key: HBASE-5272 URL: https://issues.apache.org/jira/browse/HBASE-5272 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu Currently the body of DefaultLoadBalancer.balanceCluster() is 250 lines long. It involves multiple iterations which deal with various kinds of corner cases. We should simplify this part of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5258) Move coprocessors set out of RegionLoad
Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5248) TestFSUtils#testcomputeHDFSBlocksDistribution fails intermittently
TestFSUtils#testcomputeHDFSBlocksDistribution fails intermittently
--
Key: HBASE-5248
URL: https://issues.apache.org/jira/browse/HBASE-5248
Project: HBase
Issue Type: Test
Reporter: Zhihong Yu
As can be seen from
https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK/2644/testReport/org.apache.hadoop.hbase.util/TestFSUtils/testcomputeHDFSBlocksDistribution/:
{code}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at
org.apache.hadoop.hbase.util.TestFSUtils.testcomputeHDFSBlocksDistribution(TestFSUtils.java:142)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5256) Use WritableUtils.readVInt() in RegionLoad.readFields()
Use WritableUtils.readVInt() in RegionLoad.readFields() --- Key: HBASE-5256 URL: https://issues.apache.org/jira/browse/HBASE-5256 Project: HBase Issue Type: Task Reporter: Zhihong Yu Fix For: 0.94.0 Currently in.readInt() is used in RegionLoad.readFields() More metrics would be added to RegionLoad in the future, we should utilize WritableUtils.readVInt() to reduce the amount of data exchanged between Master and region servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5242) Provide link on Master UI to hide All Monitored Tasks
Provide link on Master UI to hide All Monitored Tasks - Key: HBASE-5242 URL: https://issues.apache.org/jira/browse/HBASE-5242 Project: HBase Issue Type: Bug Reporter: Zhihong Yu Currently there is no link on Master UI to hide All Monitored Tasks. We should provide such link. Same applies to 'non-RPC Tasks', 'RPC Handler Tasks', etc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5231) Backport HBASE-3373 to 0.90
Backport HBASE-3373 to 0.90 --- Key: HBASE-5231 URL: https://issues.apache.org/jira/browse/HBASE-5231 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu This JIRA backports per-table load balancing to 0.90 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5219) Improve region selection from overloaded servers
Improve region selection from overloaded servers Key: HBASE-5219 URL: https://issues.apache.org/jira/browse/HBASE-5219 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu As Jinchao pointed out in HBASE-3724, the current region selection method still tends to move many hot regions onto a new server joining the cluster. This JIRA improves the selection to avoid such scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5223) TestMetaReaderEditor is missing call to CatalogTracker.stop()
TestMetaReaderEditor is missing call to CatalogTracker.stop() - Key: HBASE-5223 URL: https://issues.apache.org/jira/browse/HBASE-5223 Project: HBase Issue Type: Test Reporter: Zhihong Yu Attachments: 5223.txt I noticed that TestMetaReaderEditor hung on 0.92 Jenkins builds - see https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/249/console It turns out that CatalogTracker.stop() is missing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5206) Port HBASE-5155 to 0.92 and TRUNK
Port HBASE-5155 to 0.92 and TRUNK - Key: HBASE-5206 URL: https://issues.apache.org/jira/browse/HBASE-5206 Project: HBase Issue Type: Bug Reporter: Zhihong Yu This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted) to 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5194) Backport HBASE-4465 (Lazy-seek optimization for StoreFile scanners) to 0.92
Backport HBASE-4465 (Lazy-seek optimization for StoreFile scanners) to 0.92 --- Key: HBASE-5194 URL: https://issues.apache.org/jira/browse/HBASE-5194 Project: HBase Issue Type: Task Reporter: Zhihong Yu Lazy-seek optimization for StoreFile scanners is important feature This JIRA backports the feature to 0.92 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5183) Render the monitored tasks as a treeview
Render the monitored tasks as a treeview Key: HBASE-5183 URL: https://issues.apache.org/jira/browse/HBASE-5183 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu Andy made the suggestion here: https://issues.apache.org/jira/browse/HBASE-5174?focusedCommentId=13184571page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13184571 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5172) HTableInterface should extend java.io.Closeable
HTableInterface should extend java.io.Closeable --- Key: HBASE-5172 URL: https://issues.apache.org/jira/browse/HBASE-5172 Project: HBase Issue Type: Bug Reporter: Zhihong Yu Ioan Eugen Stan found this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5175) Add DoubleColumnInterpreter
Add DoubleColumnInterpreter --- Key: HBASE-5175 URL: https://issues.apache.org/jira/browse/HBASE-5175 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu DoubleColumnInterpreter was requested by Royston Sellman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5136) Redundant MonitoredTask instances in case of distributed log splitting retry
Redundant MonitoredTask instances in case of distributed log splitting retry
Key: HBASE-5136
URL: https://issues.apache.org/jira/browse/HBASE-5136
Project: HBase
Issue Type: Task
Reporter: Zhihong Yu
In case of log splitting retry, the following code would be executed multiple
times:
{code}
public long splitLogDistributed(final ListPath logDirs) throws IOException {
MonitoredTask status = TaskMonitor.get().createStatus(
Doing distributed log split in + logDirs);
{code}
leading to multiple MonitoredTask instances.
User may get confused by multiple distributed log splitting entries for the
same region server on master UI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5139) Compute (weighted) median using AggregateProtocol
Compute (weighted) median using AggregateProtocol - Key: HBASE-5139 URL: https://issues.apache.org/jira/browse/HBASE-5139 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu Suppose cf:cq1 stores numeric values and optionally cf:cq2 stores weights. This task finds out the median value among the values of cf:cq1 (See http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/R.basic/html/weighted.median.html) This can be done in two passes. The first pass utilizes AggregateProtocol where the following tuple is returned from each region: (start-rowkey, partial-sum-of-values, partial-sum-of-weights) The start-rowkey is used to sort the tuples. This way we can determine which region (called R) contains the (weighted) median. partial-sum-of-weights can be 0 if unweighted median is sought The second pass involves scanning the region R and computing partial (weighted) sum until the threshold of S/2 is crossed. The (weighted) median is returned. However, this approach wouldn't work if there is mutation in the underlying table between pass one and pass two. In that case, sequential scanning seems to be the solution which is slower than the above approach. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5133) Support zookeeper cluster key related parameters for LoadTestTool
Support zookeeper cluster key related parameters for LoadTestTool - Key: HBASE-5133 URL: https://issues.apache.org/jira/browse/HBASE-5133 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu HBASE-5124 adds zookeeper cluster key related parameters for LoadTestTool This task ports them to TRUNK. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5122) Provide flexible method for loading ColumnInterpreters
Provide flexible method for loading ColumnInterpreters
--
Key: HBASE-5122
URL: https://issues.apache.org/jira/browse/HBASE-5122
Project: HBase
Issue Type: Task
Reporter: Zhihong Yu
See the discussion on user list entitled 'AggregateProtocol Help'
From Royston:
{noformat}
I re-created my HBase table to contain Bytes.toBytes(Long) values and that
fixed it.
{noformat}
It was not the intention when AggregateProtocol was designed that users have to
change their schema to match LongColumnInterpreter.
This JIRA aims to provide a flexible way for users to load their custom
ColumnInterpreters into region servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5123) Provide more aggregate functions for Aggregations Protocol
Provide more aggregate functions for Aggregations Protocol -- Key: HBASE-5123 URL: https://issues.apache.org/jira/browse/HBASE-5123 Project: HBase Issue Type: Improvement Reporter: Zhihong Yu Royston requested the following aggregates on top of what we already have: Median, Weighted Median, Mult See discussion entitled 'AggregateProtocol Help' on user list -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5124) Backport LoadTestTool to 0.92
Backport LoadTestTool to 0.92 - Key: HBASE-5124 URL: https://issues.apache.org/jira/browse/HBASE-5124 Project: HBase Issue Type: Task Reporter: Zhihong Yu Fix For: 0.92.0 LoadTestTool is very useful. This JIRA backports LoadTestTool to 0.92 so that users don't have to build TRUNK in order to use it against 0.92 cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5117) Make locality-aware balancing move
Make locality-aware balancing move -- Key: HBASE-5117 URL: https://issues.apache.org/jira/browse/HBASE-5117 Project: HBase Issue Type: Sub-task Reporter: Zhihong Yu Fix For: 0.94.0 HBASE-4191 aims to achieve global optimum by running MIN COST MAXIMUM FLOW solver. In the current load balancer code, we can target incremental locality awareness through each balancing movement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5113) TestDrainingServer expects round robin region assignment but misses a config parameter
TestDrainingServer expects round robin region assignment but misses a config
parameter
--
Key: HBASE-5113
URL: https://issues.apache.org/jira/browse/HBASE-5113
Project: HBase
Issue Type: Test
Reporter: Zhihong Yu
Fix For: 0.92.0, 0.94.0
From
https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK/2595/testReport/org.apache.hadoop.hbase/TestDrainingServer/org_apache_hadoop_hbase_TestDrainingServer/,
we can see that some region server didn't have any regions assigned.
{code}
// Assert that every regionserver has some regions on it.
{code}
It turns out that BulkEnabler has an internal knob whose value is false:
{code}
boolean roundRobinAssignment = this.server.getConfiguration().getBoolean(
hbase.master.enabletable.roundrobin, false);
{code}
TestDrainingServer should have set this config parameter before calling
admin.enableTable(TABLENAME)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5111) Upgrade zookeeper to 3.4.2 release
Upgrade zookeeper to 3.4.2 release -- Key: HBASE-5111 URL: https://issues.apache.org/jira/browse/HBASE-5111 Project: HBase Issue Type: Task Reporter: Zhihong Yu Zookeeper 3.4.2 has just been released. We should upgrade to this release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5107) Order Regions in Transition table by duration
Order Regions in Transition table by duration - Key: HBASE-5107 URL: https://issues.apache.org/jira/browse/HBASE-5107 Project: HBase Issue Type: Task Reporter: Zhihong Yu Currently Regions in Transition table on master-status displays regions in the order of region Id. We should provide option to sort the regions according to the duration of how long region has stayed in the same state. This would help locating stuck regions easily. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5084) Allow different HTable instances to share one ExecutorService
Allow different HTable instances to share one ExecutorService - Key: HBASE-5084 URL: https://issues.apache.org/jira/browse/HBASE-5084 Project: HBase Issue Type: Task Reporter: Zhihong Yu This came out of Lily 1.1.1 release: Use a shared ExecutorService for all HTable instances, leading to better (or actual) thread reuse -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5070) Constraints implementation and javadoc changes
Constraints implementation and javadoc changes -- Key: HBASE-5070 URL: https://issues.apache.org/jira/browse/HBASE-5070 Project: HBase Issue Type: Task Reporter: Zhihong Yu This is continuation of HBASE-4605 See Stack's comments https://reviews.apache.org/r/2579/#review3980 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5055) Build against hadoop 0.22 fails
Build against hadoop 0.22 fails
---
Key: HBASE-5055
URL: https://issues.apache.org/jira/browse/HBASE-5055
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Zhihong Yu
Priority: Blocker
I got the following when compiling TRUNK against hadoop 0.22:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile (default-compile)
on project hbase: Compilation failure: Compilation failure:
[ERROR]
/Users/zhihyu/trunk-hbase/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java:[37,39]
cannot find symbol
[ERROR] symbol : class DFSInputStream
[ERROR] location: class org.apache.hadoop.hdfs.DFSClient
[ERROR]
[ERROR]
/Users/zhihyu/trunk-hbase/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java:[109,37]
cannot find symbol
[ERROR] symbol : class DFSInputStream
[ERROR] location: class
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.WALReader.WALReaderFSDataInputStream
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5040) Secure HBase builds fail
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Fix For: 0.92.0
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5042) TestReadWriteConsistencyControl should be renamed
TestReadWriteConsistencyControl should be renamed - Key: HBASE-5042 URL: https://issues.apache.org/jira/browse/HBASE-5042 Project: HBase Issue Type: Task Affects Versions: 0.92.0 Reporter: Zhihong Yu TestReadWriteConsistencyControl tests MultiVersionConsistencyControl so it should be named TestMultiVersionConsistencyControl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4978) Reduce offheap cache percentage in direct memory
Reduce offheap cache percentage in direct memory Key: HBASE-4978 URL: https://issues.apache.org/jira/browse/HBASE-4978 Project: HBase Issue Type: Task Reporter: Zhihong Yu Currently if -XX:MaxDirectMemorySize is given as JVM parameter, off heap cache would consume 95% of it. We should leave enough room in direct memory for NIO. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
