[jira] [Created] (HBASE-9669) off-heap BloomFilter
Liang Xie created HBASE-9669: Summary: off-heap BloomFilter Key: HBASE-9669 URL: https://issues.apache.org/jira/browse/HBASE-9669 Project: HBase Issue Type: Wish Components: util Affects Versions: 0.98.0 Reporter: Liang Xie After lru block off-heap, the bloom filter is becoming one of the largest contributers on java heap. it should be reasonable to have a config to off-heap the bloom filter to alleviate gc hurt. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4811) Support reverse Scan
[
https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779726#comment-13779726
]
Anil Gupta commented on HBASE-4811:
---
Hi All,
I am running HBase0.94.6. We need this feature. Can we backport this to 0.94?
If yes, then how hard the backporting would be?
Thanks,
Anil
> Support reverse Scan
>
>
> Key: HBASE-4811
> URL: https://issues.apache.org/jira/browse/HBASE-4811
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Affects Versions: 0.20.6, 0.94.7
>Reporter: John Carrino
>Assignee: chunhui shen
> Fix For: 0.98.0
>
> Attachments: 4811-0.94-v3.txt, 4811-trunk-v10.txt,
> 4811-trunk-v5.patch, HBase-4811-0.94.3modified.txt, HBase-4811-0.94-v2.txt,
> hbase-4811-trunkv11.patch, hbase-4811-trunkv12.patch,
> hbase-4811-trunkv13.patch, hbase-4811-trunkv14.patch,
> hbase-4811-trunkv15.patch, hbase-4811-trunkv16.patch,
> hbase-4811-trunkv17.patch, hbase-4811-trunkv18.patch,
> hbase-4811-trunkv19.patch, hbase-4811-trunkv1.patch,
> hbase-4811-trunkv20.patch, hbase-4811-trunkv4.patch,
> hbase-4811-trunkv6.patch, hbase-4811-trunkv7.patch, hbase-4811-trunkv8.patch,
> hbase-4811-trunkv9.patch
>
>
> Reversed scan means scan the rows backward.
> And StartRow bigger than StopRow in a reversed scan.
> For example, for the following rows:
> aaa/c1:q1/value1
> aaa/c1:q2/value2
> bbb/c1:q1/value1
> bbb/c1:q2/value2
> ccc/c1:q1/value1
> ccc/c1:q2/value2
> ddd/c1:q1/value1
> ddd/c1:q2/value2
> eee/c1:q1/value1
> eee/c1:q2/value2
> you could do a reversed scan from 'ddd' to 'bbb'(exclude) like this:
> Scan scan = new Scan();
> scan.setStartRow('ddd');
> scan.setStopRow('bbb');
> scan.setReversed(true);
> for(Result result:htable.getScanner(scan)){
> System.out.println(result);
> }
> Aslo you could do the reversed scan with shell like this:
> hbase> scan 'table',{REVERSED => true,STARTROW=>'ddd', STOPROW=>'bbb'}
> And the output is:
> ddd/c1:q1/value1
> ddd/c1:q2/value2
> ccc/c1:q1/value1
> ccc/c1:q2/value2
> NOTE: when setting reversed as true for a client scan, you must set the start
> row, else will throw exception. Through {@link
> Scan#createBiggestByteArray(int)},you could get a big enough byte array as
> the start row
> All the documentation I find about HBase says that if you want forward and
> reverse scans you should just build 2 tables and one be ascending and one
> descending. Is there a fundamental reason that HBase only supports forward
> Scan? It seems like a lot of extra space overhead and coding overhead (to
> keep them in sync) to support 2 tables.
> I am assuming this has been discussed before, but I can't find the
> discussions anywhere about it or why it would be infeasible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9488) Improve performance for small scan
[ https://issues.apache.org/jira/browse/HBASE-9488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779710#comment-13779710 ] Lars Hofhansl commented on HBASE-9488: -- I ran this through my simple tight loop scan tester. No performance degradation, so I'm fine with committing this to 0.94. If there are no objections I will so tomorrow. > Improve performance for small scan > -- > > Key: HBASE-9488 > URL: https://issues.apache.org/jira/browse/HBASE-9488 > Project: HBase > Issue Type: Improvement > Components: Client, Performance, Scanners >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.98.0, 0.96.0 > > Attachments: hbase-9488-94-v3.patch, HBASE-9488-trunk.patch, > HBASE-9488-trunkV2.patch, HBASE-9488-trunkV3.patch, HBASE-9488-trunkV4.patch, > HBASE-9488-trunkV4.patch, HBASE-9488-trunkV5.patch, > mergeRpcCallForScan.patch, test results.jpg > > > review board: > https://reviews.apache.org/r/14059/ > *Performance Improvement* > Test shows about 1.5~3X improvement for small scan where limit<=50 under > cache hit ratio=100%. > See more performance test result from the picture attachment > *Usage:* > Scan scan = new Scan(startRow,stopRow); > scan.setSmall(true); > ResultScanner scanner = table.getScanner(scan); > Set the new 'small' attribute as true for scan object, others are the same > Now, one scan operation would call 3 RPC at least: > openScanner(); > next(); > closeScanner(); > I think we could reduce the RPC call to one for small scan to get better > performance > Also using pread is better than seek+read for small scan (For this point, see > more on HBASE-7266) > Implements such a small scan as the patch, and take the performance test as > following: > a.Environment： > patched on 0.94 version > one regionserver; > one client with 50 concurrent threads; > KV size:50/100; > 100% LRU cache hit ratio; > Random start row of scan > b.Results: > See the picture attachment > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779651#comment-13779651 ] Hudson commented on HBASE-8711: --- SUCCESS: Integrated in HBase-TRUNK #4567 (See [https://builds.apache.org/job/HBase-TRUNK/4567/]) HBASE-8711 Requests count is completely off (James Kinley via JD) (jdcryans: rev 1526754) * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: James Kinley > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-8711-1.patch, RPS-TEST-1.log, RPS-TEST-2.log > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9656) Remove decimal places from Requests Per Second stats
[ https://issues.apache.org/jira/browse/HBASE-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779652#comment-13779652 ] Hudson commented on HBASE-9656: --- SUCCESS: Integrated in HBase-TRUNK #4567 (See [https://builds.apache.org/job/HBase-TRUNK/4567/]) HBASE-9656 Remove decimal places from Requests Per Second stats (James Kinley via JD) (jdcryans: rev 1526755) * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon > Remove decimal places from Requests Per Second stats > > > Key: HBASE-9656 > URL: https://issues.apache.org/jira/browse/HBASE-9656 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 0.95.2 >Reporter: James Kinley >Assignee: James Kinley >Priority: Trivial > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9656-1.patch > > > The Requests Per Second stats on the Master and RegionServer UI pages would > look better without decimal places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779653#comment-13779653
]
Hudson commented on HBASE-9439:
---
SUCCESS: Integrated in HBase-TRUNK #4567 (See
[https://builds.apache.org/job/HBase-TRUNK/4567/])
HBASE-9439 shell command list snows something not meaningful (jmhsieh: rev
1526742)
* /hbase/trunk/hbase-shell/src/main/ruby/hbase/admin.rb
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.95.2
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9656) Remove decimal places from Requests Per Second stats
[ https://issues.apache.org/jira/browse/HBASE-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779641#comment-13779641 ] Hudson commented on HBASE-9656: --- SUCCESS: Integrated in hbase-0.96 #100 (See [https://builds.apache.org/job/hbase-0.96/100/]) HBASE-9656 Remove decimal places from Requests Per Second stats (James Kinley via JD) (jdcryans: rev 1526756) * /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon > Remove decimal places from Requests Per Second stats > > > Key: HBASE-9656 > URL: https://issues.apache.org/jira/browse/HBASE-9656 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 0.95.2 >Reporter: James Kinley >Assignee: James Kinley >Priority: Trivial > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9656-1.patch > > > The Requests Per Second stats on the Master and RegionServer UI pages would > look better without decimal places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779640#comment-13779640 ] Hudson commented on HBASE-8711: --- SUCCESS: Integrated in hbase-0.96 #100 (See [https://builds.apache.org/job/hbase-0.96/100/]) HBASE-8711 Requests count is completely off (jdcryans: rev 1526753) * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: James Kinley > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-8711-1.patch, RPS-TEST-1.log, RPS-TEST-2.log > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779643#comment-13779643
]
Hudson commented on HBASE-9439:
---
SUCCESS: Integrated in hbase-0.96 #100 (See
[https://builds.apache.org/job/hbase-0.96/100/])
HBASE-9439 shell command list shows something not meaningful (jmhsieh: rev
1526743)
* /hbase/branches/0.96/hbase-shell/src/main/ruby/hbase/admin.rb
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.95.2
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779642#comment-13779642 ] Hudson commented on HBASE-9655: --- SUCCESS: Integrated in hbase-0.96 #100 (See [https://builds.apache.org/job/hbase-0.96/100/]) HBASE-9655 IntegrationTestMTTR can loop forever on improperly configured clusters (ndimiduk: rev 1526733) * /hbase/branches/0.96/hbase-it/src/test/java/org/apache/hadoop/hbase/mttr/IntegrationTestMTTR.java > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9668) Apply small scan to Scan's used by AggregationClient
[
https://issues.apache.org/jira/browse/HBASE-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779626#comment-13779626
]
Hadoop QA commented on HBASE-9668:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12605375/9668-v1.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7395//console
This message is automatically generated.
> Apply small scan to Scan's used by AggregationClient
>
>
> Key: HBASE-9668
> URL: https://issues.apache.org/jira/browse/HBASE-9668
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9668-v1.txt
>
>
> The Scan objects used to construct AggregateRequest's for aggregates
> supported by AggregationClient qualify as small scan because response from
> each region is small.
> We should utilize small scan for better performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput
[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779607#comment-13779607 ] Liu Shaohui commented on HBASE-8755: update the result table and add ops diff ||Thread number|| Time without Patch || Ops without Patch||Time with Patch || Ops with Patch || Time diff % || Ops diff % || |1|579.38|1725.983|625.937|1597.605|-8.04|-7.44| |1|580.307|1723.226|630.346|1586.43|-8.62|-7.94| |1|577.853|1730.544|654.205|1528.573|-13.21|-11.67| |5|799.579|6253.291|785.696|6363.785|1.74|1.77| |5|795.013|6289.206|780.642|6404.984|1.81|1.84| |5|826.27|6051.291|781.909|6394.606|5.37|5.67| |50|3290.482|15195.343|1165.773|42890|64.57|182.26| |50|3298.387|15158.925|1167.992|42808.516|64.59|182.40| |50|3224.495|15506.304|1154.921|43293.004|64.18|179.20| |75|4450.76|16851.055|1253.448|59834.953|71.84|255.08| |75|4506.143|16643.945|1269.806|59064.141|71.82|254.87| |75|4516.453|16605.951|1245.954|60194.84|72.41|262.49| |100|5561.074|17982.137|1493.102|66974.656|73.15|272.45| |100|5616.81|17803.699|1496.263|66833.172|73.36|275.39| |100|5612.268|17818.107|1468.5|68096.695|73.83|282.18| Time diff = (Time without Patch - Time with Patch) / Time without Patch * 100 Ops diff = (Ops with Patch - Ops without Patch) / Ops without Patch * 100 [~stack] What are the hdfs and hbase version of your test? We may rebo the tests in cluster with same hdfs and hbase versions as yours. > A new write thread model for HLog to improve the overall HBase write > throughput > --- > > Key: HBASE-8755 > URL: https://issues.apache.org/jira/browse/HBASE-8755 > Project: HBase > Issue Type: Improvement > Components: Performance, wal >Reporter: Feng Honghua >Assignee: stack >Priority: Critical > Fix For: 0.96.1 > > Attachments: 8755trunkV2.txt, HBASE-8755-0.94-V0.patch, > HBASE-8755-0.94-V1.patch, HBASE-8755-trunk-V0.patch, HBASE-8755-trunk-V1.patch > > > In current write model, each write handler thread (executing put()) will > individually go through a full 'append (hlog local buffer) => HLog writer > append (write to hdfs) => HLog writer sync (sync hdfs)' cycle for each write, > which incurs heavy race condition on updateLock and flushLock. > The only optimization where checking if current syncTillHere > txid in > expectation for other thread help write/sync its own txid to hdfs and > omitting the write/sync actually help much less than expectation. > Three of my colleagues(Ye Hangjun / Wu Zesheng / Zhang Peng) at Xiaomi > proposed a new write thread model for writing hdfs sequence file and the > prototype implementation shows a 4X improvement for throughput (from 17000 to > 7+). > I apply this new write thread model in HLog and the performance test in our > test cluster shows about 3X throughput improvement (from 12150 to 31520 for 1 > RS, from 22000 to 7 for 5 RS), the 1 RS write throughput (1K row-size) > even beats the one of BigTable (Precolator published in 2011 says Bigtable's > write throughput then is 31002). I can provide the detailed performance test > results if anyone is interested. > The change for new write thread model is as below: > 1> All put handler threads append the edits to HLog's local pending buffer; > (it notifies AsyncWriter thread that there is new edits in local buffer) > 2> All put handler threads wait in HLog.syncer() function for underlying > threads to finish the sync that contains its txid; > 3> An single AsyncWriter thread is responsible for retrieve all the buffered > edits in HLog's local pending buffer and write to the hdfs > (hlog.writer.append); (it notifies AsyncFlusher thread that there is new > writes to hdfs that needs a sync) > 4> An single AsyncFlusher thread is responsible for issuing a sync to hdfs > to persist the writes by AsyncWriter; (it notifies the AsyncNotifier thread > that sync watermark increases) > 5> An single AsyncNotifier thread is responsible for notifying all pending > put handler threads which are waiting in the HLog.syncer() function > 6> No LogSyncer thread any more (since there is always > AsyncWriter/AsyncFlusher threads do the same job it does) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9668) Apply small scan to Scan's used by AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-9668: -- Resolution: Later Status: Resolved (was: Patch Available) > Apply small scan to Scan's used by AggregationClient > > > Key: HBASE-9668 > URL: https://issues.apache.org/jira/browse/HBASE-9668 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 9668-v1.txt > > > The Scan objects used to construct AggregateRequest's for aggregates > supported by AggregationClient qualify as small scan because response from > each region is small. > We should utilize small scan for better performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9668) Apply small scan to Scan's used by AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779583#comment-13779583 ] Ted Yu commented on HBASE-9668: --- In Phoenix, there is ScanRegionObserver#getTopNScanner(). The concept of small scan is applicable to region scanner that does TopN. > Apply small scan to Scan's used by AggregationClient > > > Key: HBASE-9668 > URL: https://issues.apache.org/jira/browse/HBASE-9668 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 9668-v1.txt > > > The Scan objects used to construct AggregateRequest's for aggregates > supported by AggregationClient qualify as small scan because response from > each region is small. > We should utilize small scan for better performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput
[
https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779581#comment-13779581
]
Liu Shaohui commented on HBASE-8755:
[~stack] [~fenghh]
We redo the comparision test using HogPE. Here are the results:
Test env:
hdfs: cdh 4.1.0, five datanode, each node has 12 sata disks.
hbase: 0.94.3
HLogPE is run on one of these datanodes, so one replica of hlog's block will be
at local datanode.
The params of HLogPE are: -iterations 100 -keySize 50 -valueSize 100, which
are same as stack's tests.
{code}
for i in 1 5 50 75 100; do
for j in 1 2 3; do
./bin/hbase
org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation -verify
-threads "${i}" -iterations 100 -keySize 50 -valueSize 100 &>
log-patch"${i}"."${j}".txt;
grep "Summary: " log-patch"${i}"."${j}".txt
done;
done
{code}
||Thread||Count||WithoutPatch||WithPatch||
|1|579.380|625.937|-8.03|
|1|580.307|630.346|-8.62|
|1|577.853|654.20|-13.21|
|5|799.579|785.696|1.73|
|5|795.013|780.642|1.80|
|5|826.270|781.909|5.36|
|50|3290.482|1165.773|64.57|
|50|3298.387|1167.992|64.58|
|50|3224.495|1154.921|64.18|
|75|4450.760|1253.448|71.83|
|75|4506.143|1269.806|71.82|
|75|4516.453|1245.954|72.41|
|100|5561.074|1493.102|73.15|
|100|5616.810|1496.263|73.36|
|100|5612.268|1468.500|73.83|
a, When thread number is 1, we see that the performance of our test is about
40% better than that of stack's test, both in old thread mode and new thread
mode. [~stack] what's the hdfs version in your test or are there special
configs?
b, When thread number is 5, we do not see -50% downgrade.
> A new write thread model for HLog to improve the overall HBase write
> throughput
> ---
>
> Key: HBASE-8755
> URL: https://issues.apache.org/jira/browse/HBASE-8755
> Project: HBase
> Issue Type: Improvement
> Components: Performance, wal
>Reporter: Feng Honghua
>Assignee: stack
>Priority: Critical
> Fix For: 0.96.1
>
> Attachments: 8755trunkV2.txt, HBASE-8755-0.94-V0.patch,
> HBASE-8755-0.94-V1.patch, HBASE-8755-trunk-V0.patch, HBASE-8755-trunk-V1.patch
>
>
> In current write model, each write handler thread (executing put()) will
> individually go through a full 'append (hlog local buffer) => HLog writer
> append (write to hdfs) => HLog writer sync (sync hdfs)' cycle for each write,
> which incurs heavy race condition on updateLock and flushLock.
> The only optimization where checking if current syncTillHere > txid in
> expectation for other thread help write/sync its own txid to hdfs and
> omitting the write/sync actually help much less than expectation.
> Three of my colleagues(Ye Hangjun / Wu Zesheng / Zhang Peng) at Xiaomi
> proposed a new write thread model for writing hdfs sequence file and the
> prototype implementation shows a 4X improvement for throughput (from 17000 to
> 7+).
> I apply this new write thread model in HLog and the performance test in our
> test cluster shows about 3X throughput improvement (from 12150 to 31520 for 1
> RS, from 22000 to 7 for 5 RS), the 1 RS write throughput (1K row-size)
> even beats the one of BigTable (Precolator published in 2011 says Bigtable's
> write throughput then is 31002). I can provide the detailed performance test
> results if anyone is interested.
> The change for new write thread model is as below:
> 1> All put handler threads append the edits to HLog's local pending buffer;
> (it notifies AsyncWriter thread that there is new edits in local buffer)
> 2> All put handler threads wait in HLog.syncer() function for underlying
> threads to finish the sync that contains its txid;
> 3> An single AsyncWriter thread is responsible for retrieve all the buffered
> edits in HLog's local pending buffer and write to the hdfs
> (hlog.writer.append); (it notifies AsyncFlusher thread that there is new
> writes to hdfs that needs a sync)
> 4> An single AsyncFlusher thread is responsible for issuing a sync to hdfs
> to persist the writes by AsyncWriter; (it notifies the AsyncNotifier thread
> that sync watermark increases)
> 5> An single AsyncNotifier thread is responsible for notifying all pending
> put handler threads which are waiting in the HLog.syncer() function
> 6> No LogSyncer thread any more (since there is always
> AsyncWriter/AsyncFlusher threads do the same job it does)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9583) add document for getShortMidpointKey
[ https://issues.apache.org/jira/browse/HBASE-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779579#comment-13779579 ] Liang Xie commented on HBASE-9583: -- attached v3 rev took [~jmhsieh]'s latest comments > add document for getShortMidpointKey > > > Key: HBASE-9583 > URL: https://issues.apache.org/jira/browse/HBASE-9583 > Project: HBase > Issue Type: Task > Components: HFile >Affects Versions: 0.98.0 >Reporter: Liang Xie >Assignee: Liang Xie > Attachments: HBase-9583.txt, HBase-9583-v2.txt, HBase-9583-v3.txt > > > add the faked key to documentation http://hbase.apache.org/book.html#hfilev2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9583) add document for getShortMidpointKey
[ https://issues.apache.org/jira/browse/HBASE-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-9583: - Attachment: HBase-9583-v3.txt > add document for getShortMidpointKey > > > Key: HBASE-9583 > URL: https://issues.apache.org/jira/browse/HBASE-9583 > Project: HBase > Issue Type: Task > Components: HFile >Affects Versions: 0.98.0 >Reporter: Liang Xie >Assignee: Liang Xie > Attachments: HBase-9583.txt, HBase-9583-v2.txt, HBase-9583-v3.txt > > > add the faked key to documentation http://hbase.apache.org/book.html#hfilev2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9668) Apply small scan to Scan's used by AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779572#comment-13779572 ] chunhui shen commented on HBASE-9668: - I don't think we should apply small scan for AggregationClient. The function of small scan: 1.Reduce RPC, but it won't take affect in AggregationClient since we call it through coprocessor 2.Using position read when reading data block from HDFS. Position read is better if scan range is small, and seek+read is better if scan range is large. Although the response is small, the scan range may be large(It means it will scan a lot of data) > Apply small scan to Scan's used by AggregationClient > > > Key: HBASE-9668 > URL: https://issues.apache.org/jira/browse/HBASE-9668 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 9668-v1.txt > > > The Scan objects used to construct AggregateRequest's for aggregates > supported by AggregationClient qualify as small scan because response from > each region is small. > We should utilize small scan for better performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9668) Apply small scan to Scan's used by AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-9668: -- Summary: Apply small scan to Scan's used by AggregationClient (was: Apply small scan to AggregationClient) > Apply small scan to Scan's used by AggregationClient > > > Key: HBASE-9668 > URL: https://issues.apache.org/jira/browse/HBASE-9668 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 9668-v1.txt > > > The Scan objects used to construct AggregateRequest's for aggregates > supported by AggregationClient qualify as small scan because response from > each region is small. > We should utilize small scan for better performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9668) Apply small scan to AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-9668: -- Status: Patch Available (was: Open) > Apply small scan to AggregationClient > - > > Key: HBASE-9668 > URL: https://issues.apache.org/jira/browse/HBASE-9668 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 9668-v1.txt > > > The Scan objects used to construct AggregateRequest's for aggregates > supported by AggregationClient qualify as small scan because response from > each region is small. > We should utilize small scan for better performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9668) Apply small scan to AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-9668: -- Attachment: 9668-v1.txt > Apply small scan to AggregationClient > - > > Key: HBASE-9668 > URL: https://issues.apache.org/jira/browse/HBASE-9668 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 9668-v1.txt > > > The Scan objects used to construct AggregateRequest's for aggregates > supported by AggregationClient qualify as small scan because response from > each region is small. > We should utilize small scan for better performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9668) Apply small scan to AggregationClient
Ted Yu created HBASE-9668: - Summary: Apply small scan to AggregationClient Key: HBASE-9668 URL: https://issues.apache.org/jira/browse/HBASE-9668 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Ted Yu The Scan objects used to construct AggregateRequest's for aggregates supported by AggregationClient qualify as small scan because response from each region is small. We should utilize small scan for better performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779555#comment-13779555 ] Hudson commented on HBASE-9655: --- SUCCESS: Integrated in HBase-TRUNK #4566 (See [https://builds.apache.org/job/HBase-TRUNK/4566/]) HBASE-9655 IntegrationTestMTTR can loop forever on improperly configured clusters (ndimiduk: rev 1526729) * /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/mttr/IntegrationTestMTTR.java > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779556#comment-13779556 ] Hudson commented on HBASE-9390: --- SUCCESS: Integrated in HBase-TRUNK #4566 (See [https://builds.apache.org/job/HBase-TRUNK/4566/]) hbase-9390: coprocessors observers are not called during a recovery with the new log replay algorithm - part2 (jeffreyz: rev 1526696) * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/Admin.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java > coprocessors observers are not called during a recovery with the new log > replay algorithm > - > > Key: HBASE-9390 > URL: https://issues.apache.org/jira/browse/HBASE-9390 > Project: HBase > Issue Type: Bug > Components: Coprocessors, MTTR >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Jeffrey Zhong > Attachments: copro.patch, hbase-9390-part2.patch, > hbase-9390-part2-v2.patch, hbase-9390.patch, hbase-9390-v2.patch > > > See the patch to reproduce the issue: If we activate log replay we don't have > the events on WAL restore. > Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9435) Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779552#comment-13779552 ] Francis Liu commented on HBASE-9435: Nick, Devaraj and I had a quick discussion over this. Here's the quick summary: Prior to this patch http requests using json representations with or without the '@' prefix were both valid. With this patch only no-'@'-prefix requests are valid. The '@' prefix style notation is known as MAPPED notation in jersey, tho we have been running with NATURAL notation (no '@' prefix) since the code was checked-in in 2009. Which seems to be a bug in jersey, which is no longer surfaced with this patch since we use jackson. Tho server responses are always represented using the NATURAL notation. This problem is further exacerbated by the fact that our stargate doc is inconsistent or wrong. Request examples use a mix of MAPPED or NATURAL notation. While some responses use the MAPPED notation. Given this situation we can either continue to support requests which use the MAPPED notation. Or we can deprecate support for that which seems cleaner and would cause less confusion IMHO. One concern here is the impact to users if we deprecate support? Thoughts? It's a given that we'll have to update the documentation. Nick has suggested we deprecate the wiki and link the book to the rest javadoc. > Fix jersey serialization/deserialization of json objects > > > Key: HBASE-9435 > URL: https://issues.apache.org/jira/browse/HBASE-9435 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Blocker > Fix For: 0.98.0, 0.96.1 > > Attachments: HBASE-9435.patch, HBASE-9435.patch > > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779542#comment-13779542 ] Hudson commented on HBASE-9655: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #764 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/764/]) HBASE-9655 IntegrationTestMTTR can loop forever on improperly configured clusters (ndimiduk: rev 1526729) * /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/mttr/IntegrationTestMTTR.java > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779548#comment-13779548
]
Hudson commented on HBASE-9610:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #764 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/764/])
HBASE-9610 TestThriftServer.testAll failing; ADDENDUM (stack: rev 1526649)
*
/hbase/trunk/hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9662) PerformanceEvaluation input do not handle tags properties
[
https://issues.apache.org/jira/browse/HBASE-9662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779546#comment-13779546
]
Hudson commented on HBASE-9662:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #764 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/764/])
HBASE-9662 PerformanceEvaluation input do not handle tags properties
(mbertozzi: rev 1526452)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
> PerformanceEvaluation input do not handle tags properties
> -
>
> Key: HBASE-9662
> URL: https://issues.apache.org/jira/browse/HBASE-9662
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.98.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0
>
> Attachments: HBASE-9662-v0.patch
>
>
> After HBASE-8496 commit, PerformanceEvaluation is throwing an exception due
> to missing tags properties on the input line.
> {code}
> java.lang.IndexOutOfBoundsException: No group 7
> at java.util.regex.Matcher.group(Matcher.java:470)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation$PeInputFormat.getSplits(PerformanceEvaluation.java:351)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9640) Increment of loadSequence in CoprocessorHost#loadInstance() is thread-unsafe
[
https://issues.apache.org/jira/browse/HBASE-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779544#comment-13779544
]
Hudson commented on HBASE-9640:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #764 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/764/])
HBASE-9640 Increment of loadSequence in CoprocessorHost#loadInstance() is
thread-unsafe (tedyu: rev 1526519)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
> Increment of loadSequence in CoprocessorHost#loadInstance() is thread-unsafe
> -
>
> Key: HBASE-9640
> URL: https://issues.apache.org/jira/browse/HBASE-9640
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 9640.txt
>
>
> {code}
> E env = createEnvironment(implClass, impl, priority, ++loadSequence,
> conf);
> {code}
> Increment of loadSequence doesn't have proper synchronization.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779543#comment-13779543
]
Hudson commented on HBASE-9439:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #764 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/764/])
HBASE-9439 shell command list snows something not meaningful (jmhsieh: rev
1526742)
* /hbase/trunk/hbase-shell/src/main/ruby/hbase/admin.rb
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.95.2
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779545#comment-13779545 ] Hudson commented on HBASE-9390: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #764 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/764/]) hbase-9390: coprocessors observers are not called during a recovery with the new log replay algorithm - part2 (jeffreyz: rev 1526696) * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/Admin.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java > coprocessors observers are not called during a recovery with the new log > replay algorithm > - > > Key: HBASE-9390 > URL: https://issues.apache.org/jira/browse/HBASE-9390 > Project: HBase > Issue Type: Bug > Components: Coprocessors, MTTR >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Jeffrey Zhong > Attachments: copro.patch, hbase-9390-part2.patch, > hbase-9390-part2-v2.patch, hbase-9390.patch, hbase-9390-v2.patch > > > See the patch to reproduce the issue: If we activate log replay we don't have > the events on WAL restore. > Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9661) Consistent log severity level guards and statements
[
https://issues.apache.org/jira/browse/HBASE-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779547#comment-13779547
]
Hudson commented on HBASE-9661:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #764 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/764/])
HBASE-9661 Consistent log severity level guards and statements (Jackie Chang)
(jmhsieh: rev 1526620)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
> Consistent log severity level guards and statements
>
>
> Key: HBASE-9661
> URL: https://issues.apache.org/jira/browse/HBASE-9661
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.95.2
>Reporter: Jackie Chang
>Assignee: Jackie Chang
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9661.patch
>
>
> A log statement should be guarded by its matching severity level. A log
> statement like
> if (LOG.isTraceEnabled()) {
>LOG.debug(identifier + " opening connection to ZooKeeper ensemble=" +
> ensemble);
> doesn't make much sense because the log message is only printed out when
> TRACE-level is enabled. This inconsistency was possibly introduced when
> developers demoted the original log statement from DEBUG but forgot to change
> its corresponding log severity level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779541#comment-13779541 ] Hudson commented on HBASE-9390: --- FAILURE: Integrated in hbase-0.96 #99 (See [https://builds.apache.org/job/hbase-0.96/99/]) hbase-9390: coprocessors observers are not called during a recovery with the new log replay algorithm - part2 (jeffreyz: rev 1526697) * /hbase/branches/0.96/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/branches/0.96/hbase-protocol/src/main/protobuf/Admin.proto * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java > coprocessors observers are not called during a recovery with the new log > replay algorithm > - > > Key: HBASE-9390 > URL: https://issues.apache.org/jira/browse/HBASE-9390 > Project: HBase > Issue Type: Bug > Components: Coprocessors, MTTR >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Jeffrey Zhong > Attachments: copro.patch, hbase-9390-part2.patch, > hbase-9390-part2-v2.patch, hbase-9390.patch, hbase-9390-v2.patch > > > See the patch to reproduce the issue: If we activate log replay we don't have > the events on WAL restore. > Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9602) Cluster can't start when log splitting at startup time and the master's web UI is refreshed a few times
[
https://issues.apache.org/jira/browse/HBASE-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779529#comment-13779529
]
Hadoop QA commented on HBASE-9602:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12605353/HBASE-9602-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
org.apache.hadoop.hbase.security.access.TestTablePermissions
org.apache.hadoop.hbase.security.access.TestAccessControlFilter
org.apache.hadoop.hbase.master.handler.TestCreateTableHandler
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed
org.apache.hadoop.hbase.master.TestMasterMetrics
org.apache.hadoop.hbase.security.access.TestNamespaceCommands
org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7393//console
This message is automatically generated.
> Cluster can't start when log splitting at startup time and the master's web
> UI is refreshed a few times
> ---
>
> Key: HBASE-9602
> URL: https://issues.apache.org/jira/browse/HBASE-9602
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jean-Daniel Cryans
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9602.jstack.rtf, HBASE-9602.patch,
> HBASE-9602-v2.patch, HBASE-9602-v3.patch
>
>
> It looks like we cannot show the master's web ui at start time when there are
> logs
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779506#comment-13779506
]
Hudson commented on HBASE-9610:
---
SUCCESS: Integrated in hbase-0.96-hadoop2 #60 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/60/])
HBASE-9610 TestThriftServer.testAll failing; ADDENDUM (stack: rev 1526650)
*
/hbase/branches/0.96/hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779503#comment-13779503
]
Hudson commented on HBASE-9439:
---
SUCCESS: Integrated in hbase-0.96-hadoop2 #60 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/60/])
HBASE-9439 shell command list shows something not meaningful (jmhsieh: rev
1526743)
* /hbase/branches/0.96/hbase-shell/src/main/ruby/hbase/admin.rb
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.95.2
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779502#comment-13779502 ] Hudson commented on HBASE-9655: --- SUCCESS: Integrated in hbase-0.96-hadoop2 #60 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/60/]) HBASE-9655 IntegrationTestMTTR can loop forever on improperly configured clusters (ndimiduk: rev 1526733) * /hbase/branches/0.96/hbase-it/src/test/java/org/apache/hadoop/hbase/mttr/IntegrationTestMTTR.java > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9661) Consistent log severity level guards and statements
[
https://issues.apache.org/jira/browse/HBASE-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779505#comment-13779505
]
Hudson commented on HBASE-9661:
---
SUCCESS: Integrated in hbase-0.96-hadoop2 #60 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/60/])
HBASE-9661 Consistent log severity level guards and statements (Jackie Chang)
(jmhsieh: rev 1526619)
*
/hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
*
/hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
> Consistent log severity level guards and statements
>
>
> Key: HBASE-9661
> URL: https://issues.apache.org/jira/browse/HBASE-9661
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.95.2
>Reporter: Jackie Chang
>Assignee: Jackie Chang
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9661.patch
>
>
> A log statement should be guarded by its matching severity level. A log
> statement like
> if (LOG.isTraceEnabled()) {
>LOG.debug(identifier + " opening connection to ZooKeeper ensemble=" +
> ensemble);
> doesn't make much sense because the log message is only printed out when
> TRACE-level is enabled. This inconsistency was possibly introduced when
> developers demoted the original log statement from DEBUG but forgot to change
> its corresponding log severity level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779504#comment-13779504 ] Hudson commented on HBASE-9390: --- SUCCESS: Integrated in hbase-0.96-hadoop2 #60 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/60/]) hbase-9390: coprocessors observers are not called during a recovery with the new log replay algorithm - part2 (jeffreyz: rev 1526697) * /hbase/branches/0.96/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/branches/0.96/hbase-protocol/src/main/protobuf/Admin.proto * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java > coprocessors observers are not called during a recovery with the new log > replay algorithm > - > > Key: HBASE-9390 > URL: https://issues.apache.org/jira/browse/HBASE-9390 > Project: HBase > Issue Type: Bug > Components: Coprocessors, MTTR >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Jeffrey Zhong > Attachments: copro.patch, hbase-9390-part2.patch, > hbase-9390-part2-v2.patch, hbase-9390.patch, hbase-9390-v2.patch > > > See the patch to reproduce the issue: If we activate log replay we don't have > the events on WAL restore. > Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9583) add document for getShortMidpointKey
[ https://issues.apache.org/jira/browse/HBASE-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779483#comment-13779483 ] Jonathan Hsieh commented on HBASE-9583: --- HFiles contain many blocks that contain a range of sorted Cells. Each cell has a key. To save IO when reading Cells, the HFile also has an index that maps a Cell's start key to the offset of the beginning of a particular block. Prior to this optimization, HBase would use the key of the first cell in each data block as the index key. In HBASE-7845, we generate a new key that is lexicographically larger than the last key of the previous block and lexicographically equal or smaller than the start key of the current block. While actual keys can potentially be very long, this "fake key" or "virtual key" can be much shorter. For example, if the stop key of previous block is "the quick brown fox", the start key of current block is "the who", we could use "the r" as our virtual key in our hfile index. There are two benefits to this: 1) having shorter keys reduces the hfile index size, (allowing us to keep more indexes in memory), and 2) using something closer to the end key of the previous block allows us to avoid a potential extra IO when the target key lives in between the "virtual key" and the key of the first element in the target block. This optimization (implemented by the getShortMidpointKey method) is inspired by LevelDB's ByteWiseComparatorImpl::FindShortestSeparator() and FindShortSuccessor(). > add document for getShortMidpointKey > > > Key: HBASE-9583 > URL: https://issues.apache.org/jira/browse/HBASE-9583 > Project: HBase > Issue Type: Task > Components: HFile >Affects Versions: 0.98.0 >Reporter: Liang Xie >Assignee: Liang Xie > Attachments: HBase-9583.txt, HBase-9583-v2.txt > > > add the faked key to documentation http://hbase.apache.org/book.html#hfilev2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9656) Remove decimal places from Requests Per Second stats
[ https://issues.apache.org/jira/browse/HBASE-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-9656: -- Resolution: Fixed Fix Version/s: 0.96.0 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks James, committed to branch and trunk. > Remove decimal places from Requests Per Second stats > > > Key: HBASE-9656 > URL: https://issues.apache.org/jira/browse/HBASE-9656 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 0.95.2 >Reporter: James Kinley >Assignee: James Kinley >Priority: Trivial > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9656-1.patch > > > The Requests Per Second stats on the Master and RegionServer UI pages would > look better without decimal places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-8711: -- Release Note: hbase.regionserver.metrics.period can now be used to configure how often the RS metrics are computed. Defaults to 5, was 15 before this patch. > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: James Kinley > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-8711-1.patch, RPS-TEST-1.log, RPS-TEST-2.log > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans resolved HBASE-8711. --- Resolution: Fixed Fix Version/s: 0.98.0 Hadoop Flags: Reviewed Thanks James, committed to branch and trunk. > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: James Kinley > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-8711-1.patch, RPS-TEST-1.log, RPS-TEST-2.log > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779468#comment-13779468 ] Elliott Clark commented on HBASE-8711: -- +1 lgtm > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: James Kinley > Fix For: 0.96.0 > > Attachments: HBASE-8711-1.patch, RPS-TEST-1.log, RPS-TEST-2.log > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9656) Remove decimal places from Requests Per Second stats
[ https://issues.apache.org/jira/browse/HBASE-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779467#comment-13779467 ] Elliott Clark commented on HBASE-9656: -- +1 lgtm > Remove decimal places from Requests Per Second stats > > > Key: HBASE-9656 > URL: https://issues.apache.org/jira/browse/HBASE-9656 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 0.95.2 >Reporter: James Kinley >Assignee: James Kinley >Priority: Trivial > Attachments: HBASE-9656-1.patch > > > The Requests Per Second stats on the Master and RegionServer UI pages would > look better without decimal places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4713) Raise debug level to warn on ExecutionException in HConnectionManager$HConnectionImplementation
[ https://issues.apache.org/jira/browse/HBASE-4713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackie Chang updated HBASE-4713: Description: The ExecutionException is logged on debug level, and it should be logged on warn. I've met the problem in the next case: - hbase.rpc.timeout = 6 - lease time on region server = 24 - started a scan that takes more than 60 seconds on the region server ==> SocketTimeoutException logged on debug Having the log level on info, the exception was not observable on the client side and it took me a while to figure out what was happening. See also: - https://issues.apache.org/jira/browse/HBASE-3154 - http://mail-archives.apache.org/mod_mbox/hbase-user/201110.mbox/%3CCANH3+J0athaCjK-ahu-A=hrzoosjyh6s_mtpzm3_qqpfrcs...@mail.gmail.com%3E was: The ExecutionException is logged on debug level, and it should be logged on warn. I've met the problem in the next case: - hbase.rpc.timeout = 6 - lease time on region server = 24 - started a scan that takes more than 60 seconds on the region server ==> SocketTimeoutException logged on debug Having the log level on info, the exception was not observable on the client side and it took me a while to figure out what was happenning. See also: - https://issues.apache.org/jira/browse/HBASE-3154 - http://mail-archives.apache.org/mod_mbox/hbase-user/201110.mbox/%3CCANH3+J0athaCjK-ahu-A=hrzoosjyh6s_mtpzm3_qqpfrcs...@mail.gmail.com%3E > Raise debug level to warn on ExecutionException in > HConnectionManager$HConnectionImplementation > --- > > Key: HBASE-4713 > URL: https://issues.apache.org/jira/browse/HBASE-4713 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.4 >Reporter: Lucian George Iordache > Fix For: 0.92.0 > > Attachments: 4713.patch, HBASE-4713-patch.txt > > > The ExecutionException is logged on debug level, and it should be logged on > warn. I've met the problem in the next case: > - hbase.rpc.timeout = 6 > - lease time on region server = 24 > - started a scan that takes more than 60 seconds on the region server ==> > SocketTimeoutException logged on debug > Having the log level on info, the exception was not observable on the client > side and it took me a while to figure out what was happening. > See also: > - https://issues.apache.org/jira/browse/HBASE-3154 > - > http://mail-archives.apache.org/mod_mbox/hbase-user/201110.mbox/%3CCANH3+J0athaCjK-ahu-A=hrzoosjyh6s_mtpzm3_qqpfrcs...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4713) Raise debug level to warn on ExecutionException in HConnectionManager$HConnectionImplementation
[ https://issues.apache.org/jira/browse/HBASE-4713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackie Chang updated HBASE-4713: Description: The ExecutionException is logged on debug level, and it should be logged on warn. I've met the problem in the next case: - hbase.rpc.timeout = 6 - lease time on region server = 24 - started a scan that takes more than 60 seconds on the region server ==> SocketTimeoutException logged on debug Having the log level on info, the exception was not observable on the client side and it took me a while to figure out what was happenning. See also: - https://issues.apache.org/jira/browse/HBASE-3154 - http://mail-archives.apache.org/mod_mbox/hbase-user/201110.mbox/%3CCANH3+J0athaCjK-ahu-A=hrzoosjyh6s_mtpzm3_qqpfrcs...@mail.gmail.com%3E was: The ExecutionException is logged on debug level, and it should be logged on warn. I've met the problem in the next case: - hbase.rpc.timeout = 6 - lease time on region server = 24 - started a scan that takes more than 60 seconds on the region server ==> SocketTimeoutException logged on debug Having the log level on info, the exception was not observable on the client side and it took me a while to figure out what was hapenning. See also: - https://issues.apache.org/jira/browse/HBASE-3154 - http://mail-archives.apache.org/mod_mbox/hbase-user/201110.mbox/%3CCANH3+J0athaCjK-ahu-A=hrzoosjyh6s_mtpzm3_qqpfrcs...@mail.gmail.com%3E > Raise debug level to warn on ExecutionException in > HConnectionManager$HConnectionImplementation > --- > > Key: HBASE-4713 > URL: https://issues.apache.org/jira/browse/HBASE-4713 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.4 >Reporter: Lucian George Iordache > Fix For: 0.92.0 > > Attachments: 4713.patch, HBASE-4713-patch.txt > > > The ExecutionException is logged on debug level, and it should be logged on > warn. I've met the problem in the next case: > - hbase.rpc.timeout = 6 > - lease time on region server = 24 > - started a scan that takes more than 60 seconds on the region server ==> > SocketTimeoutException logged on debug > Having the log level on info, the exception was not observable on the client > side and it took me a while to figure out what was happenning. > See also: > - https://issues.apache.org/jira/browse/HBASE-3154 > - > http://mail-archives.apache.org/mod_mbox/hbase-user/201110.mbox/%3CCANH3+J0athaCjK-ahu-A=hrzoosjyh6s_mtpzm3_qqpfrcs...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779466#comment-13779466
]
Hadoop QA commented on HBASE-9612:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12605355/9612v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7394//console
This message is automatically generated.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9612v2.txt, 9612.wip.txt
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779464#comment-13779464
]
stack commented on HBASE-9612:
--
Put it up on rb here: https://reviews.apache.org/r/14357/
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9612v2.txt, 9612.wip.txt
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9612:
-
Fix Version/s: 0.96.0
0.98.0
Assignee: stack
Status: Patch Available (was: Open)
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.2, 0.95.1, 0.95.0, 0.96.0
>Reporter: Benoit Sigoure
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: 9612v2.txt, 9612.wip.txt
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9612) Ability to batch edits destined to different regions
[
https://issues.apache.org/jira/browse/HBASE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9612:
-
Attachment: 9612v2.txt
Change format of MultiRequest (and MultiRequestResponse) so can take
multiple regions of edits rather than a single regions' worth. We
dropped this functionality when we went to pb.
Because a MultiRequest now has one or more regions in it rather than
just one, the new format broke the server-side priority function; it
wants to get a single region out of the method param. The new MR
doesn't have a single region any more.
So, added priority setting by client. Currently it is for the multi
call only. Eventually all calls should go this route so we can get
rid of the ugly annotation-based priority function that we now have
running on the server-side against every incoming request.
Added priority to PayloadCarryingRpcController. Add it here to get
priority into RpcClient.
Change all PayloadCarryingRpcContollers so they have a priority set
according to table that is running the operation.
Changes in HRegionServer undoing the new multi request format and
then on other side composing the response.
Made the replay call just call into the multi call.
> Ability to batch edits destined to different regions
>
>
> Key: HBASE-9612
> URL: https://issues.apache.org/jira/browse/HBASE-9612
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0, 0.95.1, 0.95.2, 0.96.0
>Reporter: Benoit Sigoure
>Priority: Critical
> Attachments: 9612v2.txt, 9612.wip.txt
>
>
> The old (pre-PB) "multi" and "multiPut" RPCs allowed one to batch edits
> destined to different regions. Seems like we've lost this ability after the
> switch to protobufs.
> The {{MultiRequest}} only contains one {{RegionSpecifier}}, and a list of
> {{MultiAction}}. The {{MultiAction}} message is contains either a single
> {{MutationProto}} or a {{Get}} (but not both – so its name is misleading as
> there is nothing "multi" about it). Also it seems redundant with
> {{MultiGetRequest}}, I'm not sure what's the point of supporting {{Get}} in
> {{MultiAction}}.
> I propose that we change {{MultiRequest}} to be a just a list of
> {{MultiAction}}, and {{MultiAction}} will contain the {{RegionSpecifier}},
> the {{bool atomic}} and a list of {{MutationProto}}. This would be a
> non-backward compatible protobuf change.
> If we want we can support mixing edits and reads, in which case we'd also add
> a list of {{Get}} in {{MultiAction}}, and we'd have support having both that
> list and the list of {{MutationProto}} set at the same time. But this is a
> bonus and can be done later (in a backward compatible manner, hence no need
> to rush on this one).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9602) Cluster can't start when log splitting at startup time and the master's web UI is refreshed a few times
[
https://issues.apache.org/jira/browse/HBASE-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779458#comment-13779458
]
Elliott Clark commented on HBASE-9602:
--
+1. Thanks for the javadoc.
> Cluster can't start when log splitting at startup time and the master's web
> UI is refreshed a few times
> ---
>
> Key: HBASE-9602
> URL: https://issues.apache.org/jira/browse/HBASE-9602
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jean-Daniel Cryans
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9602.jstack.rtf, HBASE-9602.patch,
> HBASE-9602-v2.patch, HBASE-9602-v3.patch
>
>
> It looks like we cannot show the master's web ui at start time when there are
> logs to split because we can't reach the namespace regions.
> So it means that you can't see how things are progressing without tailing the
> log while waiting on your cluster to boot up. This wasn't the case in 0.94
> See this jstack:
> {noformat}
> "606214580@qtp-2001431298-3" prio=10 tid=0x7f6ac804 nid=0x7b1 in
> Object.wait() [0x7f6aa82bf000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1416)
> - locked <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1634)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1691)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$BlockingStub.listTableDescriptorsByNamespace(MasterAdminProtos.java:35031)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.listTableDescriptorsByNamespace(HConnectionManager.java:2181)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2265)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2262)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:94)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3155)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2261)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:461)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:270)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372)
> at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:850)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnecti
[jira] [Updated] (HBASE-9602) Cluster can't start when log splitting at startup time and the master's web UI is refreshed a few times
[
https://issues.apache.org/jira/browse/HBASE-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-9602:
--
Attachment: HBASE-9602-v3.patch
Something creeped in my last patch, v3 fixes that and adds the missing javadoc.
> Cluster can't start when log splitting at startup time and the master's web
> UI is refreshed a few times
> ---
>
> Key: HBASE-9602
> URL: https://issues.apache.org/jira/browse/HBASE-9602
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jean-Daniel Cryans
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9602.jstack.rtf, HBASE-9602.patch,
> HBASE-9602-v2.patch, HBASE-9602-v3.patch
>
>
> It looks like we cannot show the master's web ui at start time when there are
> logs to split because we can't reach the namespace regions.
> So it means that you can't see how things are progressing without tailing the
> log while waiting on your cluster to boot up. This wasn't the case in 0.94
> See this jstack:
> {noformat}
> "606214580@qtp-2001431298-3" prio=10 tid=0x7f6ac804 nid=0x7b1 in
> Object.wait() [0x7f6aa82bf000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1416)
> - locked <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1634)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1691)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$BlockingStub.listTableDescriptorsByNamespace(MasterAdminProtos.java:35031)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.listTableDescriptorsByNamespace(HConnectionManager.java:2181)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2265)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2262)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:94)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3155)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2261)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:461)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:270)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372)
> at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:850)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>
[jira] [Commented] (HBASE-9583) add document for getShortMidpointKey
[ https://issues.apache.org/jira/browse/HBASE-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779452#comment-13779452 ] Jonathan Hsieh commented on HBASE-9583: --- I'm going to give it an editing pass -- fix some typos an reorganize a little. > add document for getShortMidpointKey > > > Key: HBASE-9583 > URL: https://issues.apache.org/jira/browse/HBASE-9583 > Project: HBase > Issue Type: Task > Components: HFile >Affects Versions: 0.98.0 >Reporter: Liang Xie >Assignee: Liang Xie > Attachments: HBase-9583.txt, HBase-9583-v2.txt > > > add the faked key to documentation http://hbase.apache.org/book.html#hfilev2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9666) Integration Test Driver is broken
[
https://issues.apache.org/jira/browse/HBASE-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779449#comment-13779449
]
Elliott Clark commented on HBASE-9666:
--
tarball created from trunk following the release guidelines.
> Integration Test Driver is broken
> -
>
> Key: HBASE-9666
> URL: https://issues.apache.org/jira/browse/HBASE-9666
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Elliott Clark
>
> {code}
> bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver -r
> IntegrationTestSendTraceRequests
> {code}
> Results in :
> {code}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/hamcrest/SelfDescribing
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at org.junit.runner.Computer.getSuite(Computer.java:28)
> at org.junit.runner.Request.classes(Request.java:75)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at
> org.apache.hadoop.hbase.IntegrationTestsDriver.doWork(IntegrationTestsDriver.java:110)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> org.apache.hadoop.hbase.IntegrationTestsDriver.main(IntegrationTestsDriver.java:46)
> Caused by: java.lang.ClassNotFoundException: org.hamcrest.SelfDescribing
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 20 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9666) Integration Test Driver is broken
[
https://issues.apache.org/jira/browse/HBASE-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779445#comment-13779445
]
Enis Soztutar commented on HBASE-9666:
--
Is this from source code or tarball checkout? It works in a rpm installation
for me.
> Integration Test Driver is broken
> -
>
> Key: HBASE-9666
> URL: https://issues.apache.org/jira/browse/HBASE-9666
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Elliott Clark
>
> {code}
> bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver -r
> IntegrationTestSendTraceRequests
> {code}
> Results in :
> {code}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/hamcrest/SelfDescribing
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at org.junit.runner.Computer.getSuite(Computer.java:28)
> at org.junit.runner.Request.classes(Request.java:75)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at
> org.apache.hadoop.hbase.IntegrationTestsDriver.doWork(IntegrationTestsDriver.java:110)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> org.apache.hadoop.hbase.IntegrationTestsDriver.main(IntegrationTestsDriver.java:46)
> Caused by: java.lang.ClassNotFoundException: org.hamcrest.SelfDescribing
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 20 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9602) Cluster can't start when log splitting at startup time and the master's web UI is refreshed a few times
[
https://issues.apache.org/jira/browse/HBASE-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779439#comment-13779439
]
Hadoop QA commented on HBASE-9602:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12605350/HBASE-9602-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7391//console
This message is automatically generated.
> Cluster can't start when log splitting at startup time and the master's web
> UI is refreshed a few times
> ---
>
> Key: HBASE-9602
> URL: https://issues.apache.org/jira/browse/HBASE-9602
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jean-Daniel Cryans
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9602.jstack.rtf, HBASE-9602.patch,
> HBASE-9602-v2.patch
>
>
> It looks like we cannot show the master's web ui at start time when there are
> logs to split because we can't reach the namespace regions.
> So it means that you can't see how things are progressing without tailing the
> log while waiting on your cluster to boot up. This wasn't the case in 0.94
> See this jstack:
> {noformat}
> "606214580@qtp-2001431298-3" prio=10 tid=0x7f6ac804 nid=0x7b1 in
> Object.wait() [0x7f6aa82bf000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1416)
> - locked <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1634)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1691)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$BlockingStub.listTableDescriptorsByNamespace(MasterAdminProtos.java:35031)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.listTableDescriptorsByNamespace(HConnectionManager.java:2181)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2265)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2262)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:94)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3155)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2261)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:461)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:270)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372)
> at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:850)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.je
[jira] [Updated] (HBASE-9514) Prevent region from assigning before log splitting is done
[ https://issues.apache.org/jira/browse/HBASE-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9514: --- Fix Version/s: 0.96.0 I think this should be a release blocker. Marking it as such. > Prevent region from assigning before log splitting is done > -- > > Key: HBASE-9514 > URL: https://issues.apache.org/jira/browse/HBASE-9514 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Blocker > Fix For: 0.96.0 > > Attachments: trunk-9514_v1.patch, trunk-9514_v2.patch, > trunk-9514_v3.patch > > > If a region is assigned before log splitting is done by the server shutdown > handler, the edits belonging to this region in the hlogs of the dead server > will be lost. > Generally this is not an issue if users don't assign/unassign a region from > hbase shell or via hbase admin. These commands are marked for experts only in > the hbase shell help too. However, chaos monkey doesn't care. > If we can prevent from assigning such regions in a bad time, it would make > things a little safer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-9439:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
thanks for the review. committed 0.96 and trunk.
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.95.2
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779425#comment-13779425 ] Nick Dimiduk commented on HBASE-9655: - For what it's worth, the 6 hour runtime business appears related to HBASE-9665. > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9602) Cluster can't start when log splitting at startup time and the master's web UI is refreshed a few times
[
https://issues.apache.org/jira/browse/HBASE-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-9602:
--
Attachment: HBASE-9602-v2.patch
The first patch had a major flaw, the start() can timeout after a minute and
then {{started}} would never get set. Also it had a findbugs warning.
In this new patch I basically just did a bunch of refactoring to be able to do
lightweight checks and also not block on synchronization. I also cleaned up the
original code a bit and added a few comments.
> Cluster can't start when log splitting at startup time and the master's web
> UI is refreshed a few times
> ---
>
> Key: HBASE-9602
> URL: https://issues.apache.org/jira/browse/HBASE-9602
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.0
>Reporter: Jean-Daniel Cryans
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9602.jstack.rtf, HBASE-9602.patch,
> HBASE-9602-v2.patch
>
>
> It looks like we cannot show the master's web ui at start time when there are
> logs to split because we can't reach the namespace regions.
> So it means that you can't see how things are progressing without tailing the
> log while waiting on your cluster to boot up. This wasn't the case in 0.94
> See this jstack:
> {noformat}
> "606214580@qtp-2001431298-3" prio=10 tid=0x7f6ac804 nid=0x7b1 in
> Object.wait() [0x7f6aa82bf000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1416)
> - locked <0xbc0c1460> (a
> org.apache.hadoop.hbase.ipc.RpcClient$Call)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1634)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1691)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$BlockingStub.listTableDescriptorsByNamespace(MasterAdminProtos.java:35031)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.listTableDescriptorsByNamespace(HConnectionManager.java:2181)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2265)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2262)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:94)
> - locked <0xc09baf20> (a
> org.apache.hadoop.hbase.client.RpcRetryingCaller)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3155)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2261)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:461)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:270)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
> at
> org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372)
> at
> org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:850)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCo
[jira] [Commented] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779426#comment-13779426
]
Elliott Clark commented on HBASE-9439:
--
+1 pretty pretty.
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.95.2
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-9439:
--
Affects Version/s: 0.95.2
Hadoop Flags: (was: Reviewed)
Status: Patch Available (was: Reopened)
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.95.2
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779422#comment-13779422
]
Jonathan Hsieh commented on HBASE-9439:
---
Old:
{code}
hbase(main):012:0* list
TABLE
foo
1 row(s) in 0.0280 seconds
=> #<#:0x575e47b8>
hbase(main):013:0>
{code}
New:
{code}
hbase(main):009:0* ts = list
TABLE
foo
1 row(s) in 0.0260 seconds
=> ["foo"]
hbase(main):012:0> ts
=> ["foo"]
hbase(main):014:0> ts.each {|t| print "Table: " + t + "\n"}
Table: foo
=> ["foo"]
hbase(main):015:0>
{code}
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-9439:
--
Attachment: hbase-9439.v2.patch
This simple change makes the old output more meaning full (converts to jruby
array instead of Java List) and preserves the same behavior.
> shell command list shows something not meaningful
> -
>
> Key: HBASE-9439
> URL: https://issues.apache.org/jira/browse/HBASE-9439
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Jimmy Xiang
>Assignee: Jonathan Hsieh
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: hbase-9439.v2.patch, trunk-9439.patch
>
>
> Here is a sample output:
> {noformat}
> hbase(main):004:0> list
> TABLE
>
>
> usertable
>
>
> 1 row(s) in 0.3240 seconds
> => #<#:0x5eb8f6d>
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9655: Resolution: Fixed Fix Version/s: 0.96.0 0.98.0 Status: Resolved (was: Patch Available) Thanks for the reviews Elliott, Stack. > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779411#comment-13779411 ] Nick Dimiduk commented on HBASE-9655: - That's the plan, yes. Sound good, boss? > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779410#comment-13779410
]
Hudson commented on HBASE-9610:
---
FAILURE: Integrated in HBase-TRUNK #4565 (See
[https://builds.apache.org/job/HBase-TRUNK/4565/])
HBASE-9610 TestThriftServer.testAll failing; ADDENDUM (stack: rev 1526649)
*
/hbase/trunk/hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779408#comment-13779408 ] stack commented on HBASE-9655: -- This going into 0.96? +1 there. > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9605) Allow AggregationClient to skip specifying column family for row count aggregate
[
https://issues.apache.org/jira/browse/HBASE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779405#comment-13779405
]
Himanshu Vashishtha commented on HBASE-9605:
+1. You removed addColumn calls I see.
> Allow AggregationClient to skip specifying column family for row count
> aggregate
>
>
> Key: HBASE-9605
> URL: https://issues.apache.org/jira/browse/HBASE-9605
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9605-v1.txt
>
>
> For rowcounter job, column family is not required as input parameter.
> AggregationClient requires the specification of one column family:
> {code}
> } else if (scan.getFamilyMap().size() != 1) {
> throw new IOException("There must be only one family.");
> }
> {code}
> We should relax the above requirement for row count aggregate where
> FirstKeyOnlyFilter would be automatically applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9655) IntegrationTestMTTR can loop forever on improperly configured clusters
[ https://issues.apache.org/jira/browse/HBASE-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779398#comment-13779398 ] Nick Dimiduk commented on HBASE-9655: - I'm going to commit this momentarily. Speak now or forever hold your peace :) > IntegrationTestMTTR can loop forever on improperly configured clusters > -- > > Key: HBASE-9655 > URL: https://issues.apache.org/jira/browse/HBASE-9655 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-9655.00.patch, HBASE-9655.01.patch > > > IntegrationTestMTTR has a retry loop that can run infinitely. For instance, > running the test on a secure cluster as a user who lacks permissions to > perform table actions can cause the this scenario. Add another loop counter > and bail when a TimingCalable instance throws too many unexpected Exceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9548) Cleanup SnapshotTestingUtils
[
https://issues.apache.org/jira/browse/HBASE-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779378#comment-13779378
]
Jonathan Hsieh commented on HBASE-9548:
---
Seems funny to not close this -- does util have an implicit close?
{code}
@After
public void tearDown() throws Exception {
-admin.close();
+SnapshotTestingUtils.deleteAllSnapshots(admin);
}
{code}
nit: Add comment about what false is so reader doesn't have to hunt it down?
{code}
SnapshotTestingUtils.createSnapshotAndValidate(admin, originalTableName,
- familiesWithDataList, emptyFamiliesList, snapshotNameAsString, rootDir,
fs);
+ familiesWithDataList, emptyFamiliesList, snapshotNameAsString, rootDir,
fs, false);
{code}
> Cleanup SnapshotTestingUtils
>
>
> Key: HBASE-9548
> URL: https://issues.apache.org/jira/browse/HBASE-9548
> Project: HBase
> Issue Type: Bug
> Components: snapshots, test
>Affects Versions: 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 0.96.1
>
> Attachments: HBASE-9548-v0.patch
>
>
> Remove some duplicated code in SnapshotTestingUtils, use existing helpers and
> fix some stuff
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779334#comment-13779334
]
Hudson commented on HBASE-9610:
---
SUCCESS: Integrated in hbase-0.96 #98 (See
[https://builds.apache.org/job/hbase-0.96/98/])
HBASE-9610 TestThriftServer.testAll failing; ADDENDUM (stack: rev 1526650)
*
/hbase/branches/0.96/hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9648) collection one expired storefile causes it to be replaced by another expired storefile
[ https://issues.apache.org/jira/browse/HBASE-9648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779314#comment-13779314 ] Lars Hofhansl commented on HBASE-9648: -- [~sershe], are you agreeing? Can't quite parse that out :) Basically what I was saying was that we skip the early delete optimization for empty storefiles. It just means they'll get deleted after the compaction, but apparently without the issue we're seeing here. > collection one expired storefile causes it to be replaced by another expired > storefile > -- > > Key: HBASE-9648 > URL: https://issues.apache.org/jira/browse/HBASE-9648 > Project: HBase > Issue Type: Bug > Components: Compaction >Reporter: Sergey Shelukhin >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-9648-v0-0.94.patch, HBASE-9648-v0-trunk.patch, > HBASE-9648-v1-trunk.patch > > > There's a shortcut in compaction selection that causes the selection of > expired store files to quickly delete. > However, there's also the code that ensures we write at least one file to > preserve seqnum. This new empty file is "expired", because it has no data, > presumably. > So it's collected again, etc. > This affects 94, probably also 96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9666) Integration Test Driver is broken
[
https://issues.apache.org/jira/browse/HBASE-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elliott Clark updated HBASE-9666:
-
Affects Version/s: 0.96.0
0.98.0
> Integration Test Driver is broken
> -
>
> Key: HBASE-9666
> URL: https://issues.apache.org/jira/browse/HBASE-9666
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Elliott Clark
>
> {code}
> bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver -r
> IntegrationTestSendTraceRequests
> {code}
> Results in :
> {code}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/hamcrest/SelfDescribing
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at org.junit.runner.Computer.getSuite(Computer.java:28)
> at org.junit.runner.Request.classes(Request.java:75)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> at
> org.apache.hadoop.hbase.IntegrationTestsDriver.doWork(IntegrationTestsDriver.java:110)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> org.apache.hadoop.hbase.IntegrationTestsDriver.main(IntegrationTestsDriver.java:46)
> Caused by: java.lang.ClassNotFoundException: org.hamcrest.SelfDescribing
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 20 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9667) NullOutputStream removed from Guava 15
Matthew Greenfield created HBASE-9667:
-
Summary: NullOutputStream removed from Guava 15
Key: HBASE-9667
URL: https://issues.apache.org/jira/browse/HBASE-9667
Project: HBase
Issue Type: Improvement
Reporter: Matthew Greenfield
Priority: Minor
{{com.google.common.io.NullOutputStream}} was dropped in Guava 15.0 in favor of
{{com.google.common.io.ByteStreams.nullOutputStream()}} which prevents projects
on this artifact from upgrading from Guava 14 to Guava 15.
{noformat}
ERROR 2013-09-26 17:46:12,229 [hbase.master.MasterFileSystem] bootstrap
org.apache.hadoop.hbase.DroppedSnapshotException: region: -ROOT-,,0
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1608)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1482)
at
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1011)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:959)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:930)
at
org.apache.hadoop.hbase.master.MasterFileSystem.bootstrap(MasterFileSystem.java:447)
at
org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:387)
at
org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:134)
at
org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:119)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:536)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:395)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.NoClassDefFoundError: com/google/common/io/NullOutputStream
at
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.close(HFileWriterV2.java:374)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Writer.close(StoreFile.java:1283)
at
org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:836)
at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:747)
at
org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:2229)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1583)
... 11 more
Caused by: java.lang.ClassNotFoundException:
com.google.common.io.NullOutputStream
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 17 more
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9666) Integration Test Driver is broken
Elliott Clark created HBASE-9666:
Summary: Integration Test Driver is broken
Key: HBASE-9666
URL: https://issues.apache.org/jira/browse/HBASE-9666
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
{code}
bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver -r
IntegrationTestSendTraceRequests
{code}
Results in :
{code}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/hamcrest/SelfDescribing
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.junit.runner.Computer.getSuite(Computer.java:28)
at org.junit.runner.Request.classes(Request.java:75)
at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
at
org.apache.hadoop.hbase.IntegrationTestsDriver.doWork(IntegrationTestsDriver.java:110)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.hbase.IntegrationTestsDriver.main(IntegrationTestsDriver.java:46)
Caused by: java.lang.ClassNotFoundException: org.hamcrest.SelfDescribing
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 20 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9435) Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779223#comment-13779223 ] Devaraj Das commented on HBASE-9435: If I revert this patch, yes, it works. > Fix jersey serialization/deserialization of json objects > > > Key: HBASE-9435 > URL: https://issues.apache.org/jira/browse/HBASE-9435 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Blocker > Fix For: 0.98.0, 0.96.1 > > Attachments: HBASE-9435.patch, HBASE-9435.patch > > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779232#comment-13779232
]
Hadoop QA commented on HBASE-9610:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12605315/9610-make-large-and-catch-fail.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7390//console
This message is automatically generated.
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9664) ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
[
https://issues.apache.org/jira/browse/HBASE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779225#comment-13779225
]
Hadoop QA commented on HBASE-9664:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12605312/9664.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7389//console
This message is automatically generated.
> ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
> ---
>
> Key: HBASE-9664
> URL: https://issues.apache.org/jira/browse/HBASE-9664
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9664.txt
>
>
> In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 2
> at
> org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> {code}
> This was due to i being 1, resulting in index of 2 being used in the
> following statement:
> {code}
> KEY_SLAVE = tmp[ i+1 % 2 ];
> {code}
> See http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9435) Fix jersey serialization/deserialization of json objects
[ https://issues.apache.org/jira/browse/HBASE-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779211#comment-13779211 ] Francis Liu commented on HBASE-9435: I'll take look. Did these tests pass against a previous version of 0.96? > Fix jersey serialization/deserialization of json objects > > > Key: HBASE-9435 > URL: https://issues.apache.org/jira/browse/HBASE-9435 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Blocker > Fix For: 0.98.0, 0.96.1 > > Attachments: HBASE-9435.patch, HBASE-9435.patch > > > Stargate uses the default json marshaller/unmarshaller in natural mode. In > this mode the unmarshaller has trouble unmarshalling json instances. > This patch fixes this issue by using jackson as the marshaller/unmarshaller > instead. > I've also updated all the model unit tests to test json > serialization/deserialization. Backwards compatibilty can be verified by > modify the test base class to use the original marshaller/unmarshaller and > see that model tests pass. > The patch is backward compatible except for StorageClusterStatusModel, which > is broken anyway. It only shows one node in the liveNodes field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9661) Consistent log severity level guards and statements
[
https://issues.apache.org/jira/browse/HBASE-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779188#comment-13779188
]
Hudson commented on HBASE-9661:
---
FAILURE: Integrated in HBase-TRUNK #4564 (See
[https://builds.apache.org/job/HBase-TRUNK/4564/])
HBASE-9661 Consistent log severity level guards and statements (Jackie Chang)
(jmhsieh: rev 1526620)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
> Consistent log severity level guards and statements
>
>
> Key: HBASE-9661
> URL: https://issues.apache.org/jira/browse/HBASE-9661
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.95.2
>Reporter: Jackie Chang
>Assignee: Jackie Chang
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9661.patch
>
>
> A log statement should be guarded by its matching severity level. A log
> statement like
> if (LOG.isTraceEnabled()) {
>LOG.debug(identifier + " opening connection to ZooKeeper ensemble=" +
> ensemble);
> doesn't make much sense because the log message is only printed out when
> TRACE-level is enabled. This inconsistency was possibly introduced when
> developers demoted the original log statement from DEBUG but forgot to change
> its corresponding log severity level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9661) Consistent log severity level guards and statements
[
https://issues.apache.org/jira/browse/HBASE-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779164#comment-13779164
]
Hudson commented on HBASE-9661:
---
FAILURE: Integrated in hbase-0.96 #97 (See
[https://builds.apache.org/job/hbase-0.96/97/])
HBASE-9661 Consistent log severity level guards and statements (Jackie Chang)
(jmhsieh: rev 1526619)
*
/hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
*
/hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
> Consistent log severity level guards and statements
>
>
> Key: HBASE-9661
> URL: https://issues.apache.org/jira/browse/HBASE-9661
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.95.2
>Reporter: Jackie Chang
>Assignee: Jackie Chang
>Priority: Minor
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBASE-9661.patch
>
>
> A log statement should be guarded by its matching severity level. A log
> statement like
> if (LOG.isTraceEnabled()) {
>LOG.debug(identifier + " opening connection to ZooKeeper ensemble=" +
> ensemble);
> doesn't make much sense because the log message is only printed out when
> TRACE-level is enabled. This inconsistency was possibly introduced when
> developers demoted the original log statement from DEBUG but forgot to change
> its corresponding log severity level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9514) Prevent region from assigning before log splitting is done
[ https://issues.apache.org/jira/browse/HBASE-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779158#comment-13779158 ] Jimmy Xiang commented on HBASE-9514: [~jeffreyz], yes, it should cover this scenario as well. > Prevent region from assigning before log splitting is done > -- > > Key: HBASE-9514 > URL: https://issues.apache.org/jira/browse/HBASE-9514 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Blocker > Attachments: trunk-9514_v1.patch, trunk-9514_v2.patch, > trunk-9514_v3.patch > > > If a region is assigned before log splitting is done by the server shutdown > handler, the edits belonging to this region in the hlogs of the dead server > will be lost. > Generally this is not an issue if users don't assign/unassign a region from > hbase shell or via hbase admin. These commands are marked for experts only in > the hbase shell help too. However, chaos monkey doesn't care. > If we can prevent from assigning such regions in a bad time, it would make > things a little safer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9514) Prevent region from assigning before log splitting is done
[ https://issues.apache.org/jira/browse/HBASE-9514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779146#comment-13779146 ] Jeffrey Zhong commented on HBASE-9514: -- [~jxiang] Will your patch cover the scenario HBASE-9665 where balancer starts to move a region while the server hosting the regions dies then the region got lost due to ZK RIT state is messed up with the two concurrent assignments? > Prevent region from assigning before log splitting is done > -- > > Key: HBASE-9514 > URL: https://issues.apache.org/jira/browse/HBASE-9514 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Blocker > Attachments: trunk-9514_v1.patch, trunk-9514_v2.patch, > trunk-9514_v3.patch > > > If a region is assigned before log splitting is done by the server shutdown > handler, the edits belonging to this region in the hlogs of the dead server > will be lost. > Generally this is not an issue if users don't assign/unassign a region from > hbase shell or via hbase admin. These commands are marked for experts only in > the hbase shell help too. However, chaos monkey doesn't care. > If we can prevent from assigning such regions in a bad time, it would make > things a little safer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9665) Region gets lost when balancer & SSH both trying to assign
Jeffrey Zhong created HBASE-9665:
Summary: Region gets lost when balancer & SSH both trying to
assign
Key: HBASE-9665
URL: https://issues.apache.org/jira/browse/HBASE-9665
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.96.0
Reporter: Jeffrey Zhong
Priority: Critical
In summary, a server dies and its regions are re-assigned. While right before
SSH, balancer is starting assign one region on the server to somewhere.
The balancer assignment got preempted by the SSH assignment:
{code}
2013-09-25 11:55:32,854 INFO Priority.RpcServer.handler=7,port=60020
regionserver.HRegionServer: Received CLOSE for the
region:6deb1bfefe8cbdb443084efe919fdeb7 , which we are already trying to OPEN.
Cancelling OPENING.
{code}
The SSH assignment(by GeneralBulkAssigner) failed too due to:
{code}
2013-09-25 11:55:32,927 WARN [RS_OPEN_REGION-hor15n09:60020-2]
zookeeper.ZKAssign: regionserver:60020-0x14153d449d30ad0 Attempt to transition
the unassigned node for 6deb1bfefe8cbdb443084efe919fdeb7 from
M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING failed, the server that tried to
transition was hor15n09.gq1.ygridcore.net,60020,1380109280320 not the expected
hor15n07.gq1.ygridcore.net,60020,1380109890414
{code}
In the end, the region 6deb1bfefe8cbdb443084efe919fdeb7 is lost.
Below is the master log, you can see both balancer and SSH try to assign the
region around the same time:
{code}
2013-09-25 11:55:32,731 INFO [MASTER_SERVER_OPERATIONS-hor15n05:6-4]
master.RegionStates: Transitioning {6deb1bfefe8cbdb443084efe919fdeb7
state=PENDING_CLOSE, ts=1380110132710,
server=hor15n12.gq1.ygridcore.net,60020,1380109596307} will be handled by SSH
for hor15n12.gq1.ygridcore.net,60020,1380109596307
...
2013-09-25 11:55:32,849 INFO
[hor15n05.gq1.ygridcore.net,6,1380108611483-BalancerChore]
master.RegionStates: Transitioned {6deb1bfefe8cbdb443084efe919fdeb7
state=OFFLINE, ts=1380110132768, server=null} to
{6deb1bfefe8cbdb443084efe919fdeb7 state=PENDING_OPEN, ts=1380110132849,
server=hor15n07.gq1.ygridcore.net,60020,1380109890414}
...
2013-09-25 11:55:32,898 INFO
[hor15n05.gq1.ygridcore.net,6,1380108611483-GeneralBulkAssigner-1]
master.RegionStates: Transitioned {6deb1bfefe8cbdb443084efe919fdeb7
state=OFFLINE, ts=1380110132861, server=null} to
{6deb1bfefe8cbdb443084efe919fdeb7 state=PENDING_OPEN, ts=1380110132898,
server=hor15n09.gq1.ygridcore.net,60020,1380109280320}
{code}
Since SSH force region assignment while it doesn't recreate offline znode, the
later region opening would fail with the following error. I'm suggesting to
recreate offline znode when we force a region assignment(forceNewPlan=true)
with low impact.
{code}
2013-09-25 11:55:32,927 WARN [RS_OPEN_REGION-hor15n09:60020-2]
zookeeper.ZKAssign: regionserver:60020-0x14153d449d30ad0 Attempt to transition
the unassigned node for 6deb1bfefe8cbdb443084efe919fdeb7 from
M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING failed, the server that tried to
transition was hor15n09.gq1.ygridcore.net,60020,1380109280320 not the expected
hor15n07.gq1.ygridcore.net,60020,1380109890414
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9663) PerformanceEvaluation does not properly honor specified table name parameter
[
https://issues.apache.org/jira/browse/HBASE-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779128#comment-13779128
]
Hadoop QA commented on HBASE-9663:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12605287/HBASE-9663-v0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift.TestThriftServer
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7388//console
This message is automatically generated.
> PerformanceEvaluation does not properly honor specified table name parameter
>
>
> Key: HBASE-9663
> URL: https://issues.apache.org/jira/browse/HBASE-9663
> Project: HBase
> Issue Type: Bug
> Components: Client, test
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Aleksandr Shulman
>Assignee: Matteo Bertozzi
> Fix For: 0.95.2, 0.96.1
>
> Attachments: HBASE-9663-v0.patch
>
>
> Expected behavior:
> A user should be able to specify a given table for PerformanceEvaluation and
> have that table be used. That table does not need to exist. If it doesn't
> exist, PE will create it.
> Observed behavior:
> In creating the job, PE will use the new table name. However, the map tasks
> will fail because they are still looking for TestTable, which is not there.
> Potential causes:
> In the PE code, we see that the table's is not argument to MR:
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java#L723
> Command:
> {code} hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=t2
> sequentialWrite 2{code}
> Output:
> {code}initiating session
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Session establishment complete
> on server /10.20.187.137:2181, sessionid = 0x14159256f9b0031,
> negotiated timeout = 18
> 13/09/26 00:36:02 DEBUG client.HConnectionManager$HConnectionImplementation:
> Looked up root region location,
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@c8d427c;
> serverName=,60020,1380180157520
> 13/09/26 00:36:02 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for .META.,,1.
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779126#comment-13779126
]
Nicolas Liochon commented on HBASE-9610:
bq. only looking at the configs., it looks to me that medium setup is same as
large (correct me if I have it wrong).
Yes. The only difference between medium & large is that medium are executed by
default while large are executed with -PrunAllTests
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779123#comment-13779123
]
stack commented on HBASE-9610:
--
Committed 9610-make-large-and-catch-fail.txt to 0.96 and trunk. Lets see how
it goes.
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779120#comment-13779120
]
stack commented on HBASE-9610:
--
Chatted w/ Elliott. Not sure why the new fail. One suggestion was that we
change the test to large rather than medium (Test runs for 90seconds when
medium is supposed to be no more than 50seconds) thinking that it'd run in its
own VM avoiding concurrent cluster if one running in same JVM ... only looking
at the configs., it looks to me that medium setup is same as large (correct me
if I have it wrong).
So adding a patch that just catches this new exception in case it fails. Also
setting test to be large.
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9610) TestThriftServer.testAll failing
[
https://issues.apache.org/jira/browse/HBASE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9610:
-
Attachment: 9610-make-large-and-catch-fail.txt
> TestThriftServer.testAll failing
>
>
> Key: HBASE-9610
> URL: https://issues.apache.org/jira/browse/HBASE-9610
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Elliott Clark
> Attachments: 9610-debugging.txt, 9610_hack_fix.txt,
> 9610-make-large-and-catch-fail.txt, more_debugging.txt
>
>
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/org.apache.hbase$hbase-thrift/testReport/junit/org.apache.hadoop.hbase.thrift/TestThriftServer/testAll/
> {code}
> java.lang.AssertionError: Metrics Counters should be equal expected:<2> but
> was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> {code}
> Here too:
> http://jenkins-public.iridiant.net/job/hbase-0.96/134/
> http://jenkins-public.iridiant.net/job/hbase-0.96-hadoop2/140/
> Mind taking a looksee [~eclark]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9664) ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
[
https://issues.apache.org/jira/browse/HBASE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9664:
--
Attachment: 9664.txt
> ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
> ---
>
> Key: HBASE-9664
> URL: https://issues.apache.org/jira/browse/HBASE-9664
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9664.txt
>
>
> In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 2
> at
> org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> {code}
> This was due to i being 1, resulting in index of 2 being used in the
> following statement:
> {code}
> KEY_SLAVE = tmp[ (i+1) % 2 ];
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9664) ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
[
https://issues.apache.org/jira/browse/HBASE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9664:
--
Description:
In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
{code}
java.lang.ArrayIndexOutOfBoundsException: 2
at
org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
{code}
This was due to i being 1, resulting in index of 2 being used in the following
statement:
{code}
KEY_SLAVE = tmp[ i+1 % 2 ];
{code}
See http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html
was:
In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
{code}
java.lang.ArrayIndexOutOfBoundsException: 2
at
org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
{code}
This was due to i being 1, resulting in index of 2 being used in the following
statement:
{code}
KEY_SLAVE = tmp[ (i+1) % 2 ];
{code}
> ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
> ---
>
> Key: HBASE-9664
> URL: https://issues.apache.org/jira/browse/HBASE-9664
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9664.txt
>
>
> In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 2
> at
> org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> {code}
> This was due to i being 1, resulting in index of 2 being used in the
> following statement:
> {code}
> KEY_SLAVE = tmp[ i+1 % 2 ];
> {code}
> See http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9664) ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
[
https://issues.apache.org/jira/browse/HBASE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9664:
--
Status: Patch Available (was: Open)
> ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
> ---
>
> Key: HBASE-9664
> URL: https://issues.apache.org/jira/browse/HBASE-9664
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9664.txt
>
>
> In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 2
> at
> org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> {code}
> This was due to i being 1, resulting in index of 2 being used in the
> following statement:
> {code}
> KEY_SLAVE = tmp[ (i+1) % 2 ];
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9664) ArrayIndexOutOfBoundsException may be thrown in TestZKSecretWatcher
Ted Yu created HBASE-9664:
-
Summary: ArrayIndexOutOfBoundsException may be thrown in
TestZKSecretWatcher
Key: HBASE-9664
URL: https://issues.apache.org/jira/browse/HBASE-9664
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
In our internal Jenkins build, I saw failure in TestZKSecretWatcher:
{code}
java.lang.ArrayIndexOutOfBoundsException: 2
at
org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.setupBeforeClass(TestZKSecretWatcher.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
{code}
This was due to i being 1, resulting in index of 2 being used in the following
statement:
{code}
KEY_SLAVE = tmp[ (i+1) % 2 ];
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9648) collection one expired storefile causes it to be replaced by another expired storefile
[ https://issues.apache.org/jira/browse/HBASE-9648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779054#comment-13779054 ] Sergey Shelukhin commented on HBASE-9648: - Yeah, that is what we recommended :) Deleting the file definitely makes sense, we can shortcut and archive it immediately without any compaction. At that point there's also no need to do convoluted checks because we also have full view of all files. > collection one expired storefile causes it to be replaced by another expired > storefile > -- > > Key: HBASE-9648 > URL: https://issues.apache.org/jira/browse/HBASE-9648 > Project: HBase > Issue Type: Bug > Components: Compaction >Reporter: Sergey Shelukhin >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-9648-v0-0.94.patch, HBASE-9648-v0-trunk.patch, > HBASE-9648-v1-trunk.patch > > > There's a shortcut in compaction selection that causes the selection of > expired store files to quickly delete. > However, there's also the code that ensures we write at least one file to > preserve seqnum. This new empty file is "expired", because it has no data, > presumably. > So it's collected again, etc. > This affects 94, probably also 96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9605) Allow AggregationClient to skip specifying column family for row count aggregate
[
https://issues.apache.org/jira/browse/HBASE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779051#comment-13779051
]
Ted Yu commented on HBASE-9605:
---
testRowCountAllTable verifies that null CF allows row count result to come back.
> Allow AggregationClient to skip specifying column family for row count
> aggregate
>
>
> Key: HBASE-9605
> URL: https://issues.apache.org/jira/browse/HBASE-9605
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9605-v1.txt
>
>
> For rowcounter job, column family is not required as input parameter.
> AggregationClient requires the specification of one column family:
> {code}
> } else if (scan.getFamilyMap().size() != 1) {
> throw new IOException("There must be only one family.");
> }
> {code}
> We should relax the above requirement for row count aggregate where
> FirstKeyOnlyFilter would be automatically applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9605) Allow AggregationClient to skip specifying column family for row count aggregate
[
https://issues.apache.org/jira/browse/HBASE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779032#comment-13779032
]
Himanshu Vashishtha commented on HBASE-9605:
@Test (timeout=30)
- public void testRowCountWithNullCF() {
Why delete this? Could it be modified to show that null CF would also bring
back some result?
> Allow AggregationClient to skip specifying column family for row count
> aggregate
>
>
> Key: HBASE-9605
> URL: https://issues.apache.org/jira/browse/HBASE-9605
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 9605-v1.txt
>
>
> For rowcounter job, column family is not required as input parameter.
> AggregationClient requires the specification of one column family:
> {code}
> } else if (scan.getFamilyMap().size() != 1) {
> throw new IOException("There must be only one family.");
> }
> {code}
> We should relax the above requirement for row count aggregate where
> FirstKeyOnlyFilter would be automatically applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9663) PerformanceEvaluation does not properly honor specified table name parameter
[
https://issues.apache.org/jira/browse/HBASE-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779013#comment-13779013
]
Jonathan Hsieh commented on HBASE-9663:
---
Add quick test or present sample output?
> PerformanceEvaluation does not properly honor specified table name parameter
>
>
> Key: HBASE-9663
> URL: https://issues.apache.org/jira/browse/HBASE-9663
> Project: HBase
> Issue Type: Bug
> Components: Client, test
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Aleksandr Shulman
>Assignee: Matteo Bertozzi
> Fix For: 0.95.2, 0.96.1
>
> Attachments: HBASE-9663-v0.patch
>
>
> Expected behavior:
> A user should be able to specify a given table for PerformanceEvaluation and
> have that table be used. That table does not need to exist. If it doesn't
> exist, PE will create it.
> Observed behavior:
> In creating the job, PE will use the new table name. However, the map tasks
> will fail because they are still looking for TestTable, which is not there.
> Potential causes:
> In the PE code, we see that the table's is not argument to MR:
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java#L723
> Command:
> {code} hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=t2
> sequentialWrite 2{code}
> Output:
> {code}initiating session
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Session establishment complete
> on server /10.20.187.137:2181, sessionid = 0x14159256f9b0031,
> negotiated timeout = 18
> 13/09/26 00:36:02 DEBUG client.HConnectionManager$HConnectionImplementation:
> Looked up root region location,
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@c8d427c;
> serverName=,60020,1380180157520
> 13/09/26 00:36:02 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for .META.,,1.1028785192 is :60020
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Creating scanner over .META.
> starting at key 't2,,'
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 't2,,'
> 13/09/26 00:36:02 DEBUG catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@466e06d7
> 13/09/26 00:36:02 INFO zookeeper.ZooKeeper: Session: 0x14159256f9b0031 closed
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: EventThread shut down
> 13/09/26 00:36:02 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=:2181 sessionTimeout=18
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@c8d427c
> 13/09/26 00:36:02 INFO zookeeper.RecoverableZooKeeper: The identifier of this
> process is 10390@
> 13/09/26 00:36:02 DEBUG catalog.CatalogTracker: Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@5152cfbb
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Opening socket connection to
> server /10.20.187.137:2181. Will not attempt to authenticate using
> SASL (unknown error)
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Socket connection established to
> /10.20.187.137:2181, initiating session
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Session establishment complete
> on server /10.20.187.137:2181, sessionid = 0x14159256f9b0032,
> negotiated timeout = 18
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Creating scanner over .META.
> starting at key 't2,,'
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 't2,,'
> 13/09/26 00:36:02 DEBUG catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@5152cfbb
> 13/09/26 00:36:03 INFO zookeeper.ZooKeeper: Session: 0x14159256f9b0032 closed
> 13/09/26 00:36:03 INFO zookeeper.ClientCnxn: EventThread shut down
> 13/09/26 00:36:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing
> the arguments. Applications should implement Tool for the same.
> 13/09/26 00:36:06 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[0]
> startRow=1363147 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[1]
> startRow=1468004 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[2]
> startRow=1887432 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[3]
> startRow=209714 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.Perfor
[jira] [Updated] (HBASE-9663) PerformanceEvaluation does not properly honor specified table name parameter
[
https://issues.apache.org/jira/browse/HBASE-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-9663:
---
Fix Version/s: (was: 0.94.13)
> PerformanceEvaluation does not properly honor specified table name parameter
>
>
> Key: HBASE-9663
> URL: https://issues.apache.org/jira/browse/HBASE-9663
> Project: HBase
> Issue Type: Bug
> Components: Client, test
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Aleksandr Shulman
>Assignee: Matteo Bertozzi
> Fix For: 0.95.2, 0.96.1
>
> Attachments: HBASE-9663-v0.patch
>
>
> Expected behavior:
> A user should be able to specify a given table for PerformanceEvaluation and
> have that table be used. That table does not need to exist. If it doesn't
> exist, PE will create it.
> Observed behavior:
> In creating the job, PE will use the new table name. However, the map tasks
> will fail because they are still looking for TestTable, which is not there.
> Potential causes:
> In the PE code, we see that the table's is not argument to MR:
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java#L723
> Command:
> {code} hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=t2
> sequentialWrite 2{code}
> Output:
> {code}initiating session
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Session establishment complete
> on server /10.20.187.137:2181, sessionid = 0x14159256f9b0031,
> negotiated timeout = 18
> 13/09/26 00:36:02 DEBUG client.HConnectionManager$HConnectionImplementation:
> Looked up root region location,
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@c8d427c;
> serverName=,60020,1380180157520
> 13/09/26 00:36:02 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for .META.,,1.1028785192 is :60020
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Creating scanner over .META.
> starting at key 't2,,'
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 't2,,'
> 13/09/26 00:36:02 DEBUG catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@466e06d7
> 13/09/26 00:36:02 INFO zookeeper.ZooKeeper: Session: 0x14159256f9b0031 closed
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: EventThread shut down
> 13/09/26 00:36:02 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=:2181 sessionTimeout=18
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@c8d427c
> 13/09/26 00:36:02 INFO zookeeper.RecoverableZooKeeper: The identifier of this
> process is 10390@
> 13/09/26 00:36:02 DEBUG catalog.CatalogTracker: Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@5152cfbb
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Opening socket connection to
> server /10.20.187.137:2181. Will not attempt to authenticate using
> SASL (unknown error)
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Socket connection established to
> /10.20.187.137:2181, initiating session
> 13/09/26 00:36:02 INFO zookeeper.ClientCnxn: Session establishment complete
> on server /10.20.187.137:2181, sessionid = 0x14159256f9b0032,
> negotiated timeout = 18
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Creating scanner over .META.
> starting at key 't2,,'
> 13/09/26 00:36:02 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 't2,,'
> 13/09/26 00:36:02 DEBUG catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@5152cfbb
> 13/09/26 00:36:03 INFO zookeeper.ZooKeeper: Session: 0x14159256f9b0032 closed
> 13/09/26 00:36:03 INFO zookeeper.ClientCnxn: EventThread shut down
> 13/09/26 00:36:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing
> the arguments. Applications should implement Tool for the same.
> 13/09/26 00:36:06 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[0]
> startRow=1363147 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[1]
> startRow=1468004 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[2]
> startRow=1887432 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[3]
> startRow=209714 rows=104857 totalRows=2097152 clients=2 flushCommits=true
> writeToWAL=true
> 13/09/26 00:36:06 DEBUG hbase.PerformanceEvaluation: split[4]
> startRow=524285 rows=104857 totalRows=2097
