[jira] [Commented] (HBASE-5520) Support reseek() at RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231865#comment-13231865 ] Hadoop QA commented on HBASE-5520: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518777/HBASE-5520_4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 161 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1216//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1216//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1216//console This message is automatically generated. > Support reseek() at RegionScanner > - > > Key: HBASE-5520 > URL: https://issues.apache.org/jira/browse/HBASE-5520 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.92.0 >Reporter: Anoop Sam John >Assignee: ramkrishna.s.vasudevan > Fix For: 0.96.0 > > Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch, > HBASE-5520_3.patch, HBASE-5520_4.patch > > > reseek() is not supported currently at the RegionScanner level. We can > support the same. > This is created following the discussion under HBASE-2038 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo should compare regionId as well
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231860#comment-13231860 ] Hudson commented on HBASE-5563: --- Integrated in HBase-TRUNK-security #140 (See [https://builds.apache.org/job/HBase-TRUNK-security/140/]) HBASE-5563 Add comparison of regionId to HRegionInfo#compareTo (chunhui and jmhsieh) (Revision 1301779) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java > HRegionInfo#compareTo should compare regionId as well > - > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5575) Configure Arcanist lint engine for HBase
[ https://issues.apache.org/jira/browse/HBASE-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231862#comment-13231862 ] Hudson commented on HBASE-5575: --- Integrated in HBase-TRUNK-security #140 (See [https://builds.apache.org/job/HBase-TRUNK-security/140/]) [jira] [HBASE-5575] Configure Arcanist lint engine for HBase Summary: Adding Java lint engine to HBase's .arcconfig. The lint engine itself is part of the arc-jira repository that lives at https://github.com/facebook/arc-jira/. There are some changes to be made there to prevent lint from reporting errors for unmodified lines. Test Plan: arc lint Reviewers: nspiegelberg, Kannan, Karthik, JIRA, Liyin Reviewed By: Liyin Differential Revision: https://reviews.facebook.net/D2289 (Revision 1301751) Result = SUCCESS mbautin : Files : * /hbase/trunk/.arcconfig > Configure Arcanist lint engine for HBase > > > Key: HBASE-5575 > URL: https://issues.apache.org/jira/browse/HBASE-5575 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: Enabling-lint-2012-03-16_13_40_37.patch > > > We need to enable Arcanist lint engine in HBase, so that a commit could be > checked by running "arc lint". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5568) Multi concurrent flushcache() for one region could cause data loss
[ https://issues.apache.org/jira/browse/HBASE-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231859#comment-13231859 ] Hudson commented on HBASE-5568: --- Integrated in HBase-TRUNK-security #140 (See [https://builds.apache.org/job/HBase-TRUNK-security/140/]) HBASE-5568 Multi concurrent flushcache() for one region could cause data loss (Chunhui) (Revision 1301639) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java > Multi concurrent flushcache() for one region could cause data loss > -- > > Key: HBASE-5568 > URL: https://issues.apache.org/jira/browse/HBASE-5568 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5568-90.patch, HBASE-5568-92v2.patch, > HBASE-5568.patch, HBASE-5568.patch, HBASE-5568v2.patch > > > We could call HRegion#flushcache() concurrently now through > HRegionServer#splitRegion or HRegionServer#flushRegion by HBaseAdmin. > However, we find if HRegion#internalFlushcache() is called concurrently by > multi thread, HRegion.memstoreSize will be calculated wrong. > At the end of HRegion#internalFlushcache(), we will do > this.addAndGetGlobalMemstoreSize(-flushsize), but the flushsize may not the > actual memsize which flushed to hdfs. It cause HRegion.memstoreSize is > negative and prevent next flush if we close this region. > Logs in RS for region e9d827913a056e696c39bc569ea3 > 2012-03-11 16:31:36,690 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 128.0m > 2012-03-11 16:31:37,999 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/8162481165586107427, entries=153106, sequenceid=619316544, > memsize=59.6m, filesize=31.2m > 2012-03-11 16:31:38,830 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 134.8m > 2012-03-11 16:31:39,458 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/3425971951499794221, entries=230183, sequenceid=619316544, > memsize=68.5m, filesize=26.6m > 2012-03-11 16:31:39,459 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~128.1m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 2769ms, sequenceid=619316544, compaction > requested=false > 2012-03-11 16:31:39,459 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 6.8m > 2012-03-11 16:31:39,529 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/1811012969998104626, entries=8002, sequenceid=619332759, > memsize=3.1m, filesize=1.6m > 2012-03-11 16:31:39,640 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/770333473623552048, entries=12231, sequenceid=619332759, > memsize=3.6m, filesize=1.4m > 2012-03-11 16:31:39,641 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~134.8m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 811ms, sequenceid=619332759, compaction > requested=true > 2012-03-11 16:31:39,707 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/5656568849587368557, entries=119, sequenceid=619332979, > memsize=47.4k, filesize=25.6k > 2012-03-11 16:31:39,775 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/794343845650987521, entries=157, sequenceid=619332979, > memsize=47.8k, filesize=19.3k > 2012-03-11 16:31:39,777 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~6.8m for region > writetest1,,1331454657410.e9d827913a05 > 6e696c39bc569ea3f99f. in 318ms, sequenceid=619332979, compaction > requested=true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato
[jira] [Commented] (HBASE-5155) ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted
[
https://issues.apache.org/jira/browse/HBASE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231861#comment-13231861
]
Hudson commented on HBASE-5155:
---
Integrated in HBase-TRUNK-security #140 (See
[https://builds.apache.org/job/HBase-TRUNK-security/140/])
HBASE-5206 Port HBASE-5155 to trunk (Ashutosh Jindal) (Revision 1301709)
Result = SUCCESS
tedyu :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKTable.java
> ServerShutDownHandler And Disable/Delete should not happen parallely leading
> to recreation of regions that were deleted
> ---
>
> Key: HBASE-5155
> URL: https://issues.apache.org/jira/browse/HBASE-5155
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.4
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.90.6
>
> Attachments: HBASE-5155_1.patch, HBASE-5155_2.patch,
> HBASE-5155_3.patch, HBASE-5155_latest.patch, hbase-5155_6.patch
>
>
> ServerShutDownHandler and disable/delete table handler races. This is not an
> issue due to TM.
> -> A regionserver goes down. In our cluster the regionserver holds lot of
> regions.
> -> A region R1 has two daughters D1 and D2.
> -> The ServerShutdownHandler gets called and scans the META and gets all the
> user regions
> -> Parallely a table is disabled. (No problem in this step).
> -> Delete table is done.
> -> The tables and its regions are deleted including R1, D1 and D2.. (So META
> is cleaned)
> -> Now ServerShutdownhandler starts to processTheDeadRegion
> {code}
> if (hri.isOffline() && hri.isSplit()) {
> LOG.debug("Offlined and split region " + hri.getRegionNameAsString() +
> "; checking daughter presence");
> fixupDaughters(result, assignmentManager, catalogTracker);
> {code}
> As part of fixUpDaughters as the daughers D1 and D2 is missing for R1
> {code}
> if (isDaughterMissing(catalogTracker, daughter)) {
> LOG.info("Fixup; missing daughter " + daughter.getRegionNameAsString());
> MetaEditor.addDaughter(catalogTracker, daughter, null);
> // TODO: Log WARN if the regiondir does not exist in the fs. If its not
> // there then something wonky about the split -- things will keep going
> // but could be missing references to parent region.
> // And assign it.
> assignmentManager.assign(daughter, true);
> {code}
> we call assign of the daughers.
> Now after this we again start with the below code.
> {code}
> if (processDeadRegion(e.getKey(), e.getValue(),
> this.services.getAssignmentManager(),
> this.server.getCatalogTracker())) {
> this.services.getAssignmentManager().assign(e.getKey(), true);
> {code}
> Now when the SSH scanned the META it had R1, D1 and D2.
> So as part of the above code D1 and D2 which where assigned by fixUpDaughters
> is again assigned by
> {code}
> this.services.getAssignmentManager().assign(e.getKey(), true);
> {code}
> Thus leading to a zookeeper issue due to bad version and killing the master.
> The important part here is the regions that were deleted are recreated which
> i think is more critical.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231858#comment-13231858 ] Hudson commented on HBASE-5549: --- Integrated in HBase-TRUNK-security #140 (See [https://builds.apache.org/job/HBase-TRUNK-security/140/]) HBASE-5549 HBASE-5572 Master can fail if ZooKeeper session expires (N Keywal) (Revision 1301775) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterZKSessionRecovery.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231857#comment-13231857
]
Hudson commented on HBASE-5592:
---
Integrated in HBase-TRUNK-security #140 (See
[https://builds.apache.org/job/HBase-TRUNK-security/140/])
HBASE-5592 Make it easier to get a table from shell (Revision 1301715)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/ruby/hbase/table.rb
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5206) Port HBASE-5155 to 0.92, 0.94, and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231855#comment-13231855 ] Hudson commented on HBASE-5206: --- Integrated in HBase-TRUNK-security #140 (See [https://builds.apache.org/job/HBase-TRUNK-security/140/]) HBASE-5206 Port HBASE-5155 to trunk (Ashutosh Jindal) (Revision 1301709) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKTable.java > Port HBASE-5155 to 0.92, 0.94, and TRUNK > > > Key: HBASE-5206 > URL: https://issues.apache.org/jira/browse/HBASE-5206 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.2, 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Ashutosh Jindal > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: 5206_92_1.patch, 5206_92_latest_1.patch, > 5206_92_latest_2.patch, 5206_92_latest_3.patch, 5206_trunk_1.patch, > 5206_trunk_latest_1.patch, 5206_trunk_latest_2.patch, > 5206_trunk_latest_3.patch > > > This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should > not happen parallely leading to recreation of regions that were deleted) to > 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5581) Creating a table with invalid syntax does not give an error message when it fails
[
https://issues.apache.org/jira/browse/HBASE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231856#comment-13231856
]
Hudson commented on HBASE-5581:
---
Integrated in HBase-TRUNK-security #140 (See
[https://builds.apache.org/job/HBase-TRUNK-security/140/])
HBASE-5581 Creating a table with invalid syntax does not give an error
message when it fails (Revision 1301688)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/ruby/hbase/admin.rb
> Creating a table with invalid syntax does not give an error message when it
> fails
> -
>
> Key: HBASE-5581
> URL: https://issues.apache.org/jira/browse/HBASE-5581
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Binu John
>Priority: Minor
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5581trunk.patch, D2343.1.patch
>
>
> Creating a table with invalid syntax does not give an error message when it
> fails. In this case, it doesn't actually create the CF requested, but doesn't
> give any indication to the user that it failed.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true, NUMREGIONS
> => 20, SPLITALGO => "HexStringSplit", COMPRESSION => 'LZO', BLOOMFILTER =>
> 'ROW'}
> 0 row(s) in 3.0930 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => []}
> true
> 1 row(s) in 0.1430 seconds
>
> Putting {NUMREGIONS => 20, SPLITALGO => "HexStringSplit"} into a separate
> stanza works fine, so the feature is fine.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true,
> COMPRESSION => 'LZO', BLOOMFILTER => 'ROW'}, {NUMREGIONS => 20, SPLITALGO =>
> "HexStringSplit"}
> 0 row(s) in 2.7860 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => [{NAME => 'test', DATA_BLOCK_ENCODING =>
> 'NONE', true
> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'LZO',
> VERSIONS
> => '1', TTL => '2147483647', BLOCKSIZE => '65536', BLOOMFILTER_ERRORRATE =>
> '
> 0.01', ENCODE_ON_DISK => 'true', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}]}
>
> We should throw an error if we can't create the CF so it's clear to the user.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5572) KeeperException.SessionExpiredException management could be improved in Master
[
https://issues.apache.org/jira/browse/HBASE-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231854#comment-13231854
]
Hudson commented on HBASE-5572:
---
Integrated in HBase-TRUNK-security #140 (See
[https://builds.apache.org/job/HBase-TRUNK-security/140/])
HBASE-5549 HBASE-5572 Master can fail if ZooKeeper session expires (N
Keywal) (Revision 1301775)
Result = SUCCESS
tedyu :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterZKSessionRecovery.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java
> KeeperException.SessionExpiredException management could be improved in Master
> --
>
> Key: HBASE-5572
> URL: https://issues.apache.org/jira/browse/HBASE-5572
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5572.v1.patch, 5572.v2.patch, 5572.v2.patch,
> 5572.v2.patch
>
>
> Synthesis:
> 1) TestMasterZKSessionRecovery distinguish two cases on
> SessionExpiredException. One is explicitly not managed. However, is seems
> that there is no reason for this.
> 2) The issue lies in ActiveMasterManager#blockUntilBecomingActiveMaster, a
> quite complex function, with a useless recursive call.
> 3) TestMasterZKSessionRecovery#testMasterZKSessionRecoverySuccess is
> equivalent to TestZooKeeper#testMasterSessionExpired
> 4) TestMasterZKSessionRecovery#testMasterZKSessionRecoveryFailure can be
> removed if we merge the two cases mentioned above.
> Changes are:
> 2) Changing ActiveMasterManager#blockUntilBecomingActiveMaster to have a
> single case and remove recursion
> 1) Removing TestMasterZKSessionRecovery
> Detailed justification:
> testMasterZKSessionRecoveryFailure says:
> {noformat}
> /**
>* Negative test of master recovery from zk session expiry.
>*
>* Starts with one master. Fakes the master zk session expired.
>* Ensures the master cannot recover the expired zk session since
>* the master zk node is still there.
>*/
> public void testMasterZKSessionRecoveryFailure() throws Exception {
> MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
> HMaster m = cluster.getMaster();
> m.abort("Test recovery from zk session expired",
> new KeeperException.SessionExpiredException());
> assertTrue(m.isStopped());
> }
> {noformat}
> This tests works, i.e. the assertion is always verified.
> But do we really want this behavior?
> When looking at the code, we see that this what's happening is strange:
> - HMaster#abort calls Master#abortNow. If HMaster#abortNow returns false
> HMaster#abort stops the master.
> - HMaster#abortNow checks the exception type. As it's a
> SessionExpiredException it will try to recover, calling
> HMaster#tryRecoveringExpiredZKSession. If it cannot, it will return false
> (and that will make HMaster#abort stopping the master)
> - HMaster#tryRecoveringExpiredZKSession recreates a ZooKeeperConnection and
> then try to become the active master. If it cannot, it will return false (and
> that will make HMaster#abort stopping the master).
> - HMaster#becomeActiveMaster returns the result of
> ActiveMasterManager#blockUntilBecomingActiveMaster.
> blockUntilBecomingActiveMaster says it will return false if there is any
> error preventing it to become the active master.
> - ActiveMasterManager#blockUntilBecomingActiveMaster reads ZK for the master
> address. If it's the same port & host, it deletes the nodes, that will start
> a recursive call to blockUntilBecomingActiveMaster. This second call succeeds
> (we became the active master) and return true. This result is ignored by the
> first blockUntilBecomingActiveMaster: it return false (even if we actually
> became the active master), hence the whole suite call returns false and
> HMaster#abort stops the master.
> In oth
[jira] [Commented] (HBASE-5539) asynchbase PerformanceEvaluation
[
https://issues.apache.org/jira/browse/HBASE-5539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231853#comment-13231853
]
Benoit Sigoure commented on HBASE-5539:
---
As we discussed in person, yes the method {{testRow}} will return immediately
after firing off the RPC. That's why I have this {{latch}} variable on which I
call {{countDown()}} in the {{readCallback}}, and which I {{await()}} in the
teardown function, which is part of the code that's timed.
> asynchbase PerformanceEvaluation
>
>
> Key: HBASE-5539
> URL: https://issues.apache.org/jira/browse/HBASE-5539
> Project: HBase
> Issue Type: New Feature
> Components: performance
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Minor
> Labels: benchmark
> Attachments: 0001-asynchbase-PerformanceEvaluation.patch
>
>
> I plugged [asynchbase|https://github.com/stumbleupon/asynchbase] into
> {{PerformanceEvaluation}}. This enables testing asynchbase from
> {{PerformanceEvaluation}} and comparing its performance to {{HTable}}. Also
> asynchbase doesn't come with any benchmark, so it was good that I was able to
> plug it into {{PerformanceEvaluation}} relatively easily.
> I am in the processing of collecting results on a dev cluster running 0.92.1
> and will publish them once they're ready.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5520) Support reseek() at RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5520: -- Attachment: HBASE-5520_4.patch An udpated patch not w.r.t to the comments given by Lars. In my previous patch some commented lines were present hence removed them. Reg, startRegionOperation i go with Anoop's reply. Lars, pls share your comments on this. > Support reseek() at RegionScanner > - > > Key: HBASE-5520 > URL: https://issues.apache.org/jira/browse/HBASE-5520 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.92.0 >Reporter: Anoop Sam John >Assignee: ramkrishna.s.vasudevan > Fix For: 0.96.0 > > Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch, > HBASE-5520_3.patch, HBASE-5520_4.patch > > > reseek() is not supported currently at the RegionScanner level. We can > support the same. > This is created following the discussion under HBASE-2038 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5520) Support reseek() at RegionScanner
[
https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231843#comment-13231843
]
Anoop Sam John commented on HBASE-5520:
---
@Lars
The co processor preScannerNext() method will be getting called from the
HRegionServer
{code}
// Call coprocessor. Get region info from scanner.
HRegion region = getRegion(s.getRegionInfo().getRegionName());
if (region != null && region.getCoprocessorHost() != null) {
Boolean bypass = region.getCoprocessorHost().preScannerNext(s,
results, nbRows);
{code}
So by the time co processor calls this seek, there wont be any region op
already started. Pls correct me if I am wrong.
> Support reseek() at RegionScanner
> -
>
> Key: HBASE-5520
> URL: https://issues.apache.org/jira/browse/HBASE-5520
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 0.92.0
>Reporter: Anoop Sam John
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch,
> HBASE-5520_3.patch
>
>
> reseek() is not supported currently at the RegionScanner level. We can
> support the same.
> This is created following the discussion under HBASE-2038
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231833#comment-13231833
]
Lars Hofhansl commented on HBASE-5569:
--
This must be some strange timing issue since it *never* happens on my fast work
machine.
I think I'll revert the change until I understand this better.
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl reopened HBASE-5569:
--
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231825#comment-13231825
]
Lars Hofhansl commented on HBASE-5569:
--
Added some extra logging. Turns out that the after KV always has memstoreTS=0.
I have to conclude that this is not fixed, yet.
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231818#comment-13231818
]
Lars Hofhansl commented on HBASE-5569:
--
So I suppose this can happen when the two deletes differ only by memstoreTS.
This is a different problem from I fixed in this issue.
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5597) Findbugs check in test-patch.sh always fails
[ https://issues.apache.org/jira/browse/HBASE-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231817#comment-13231817 ] Uma Maheswara Rao G commented on HBASE-5597: Hi David, Problem with just upping the count is, later if you remove some code(contains current findbugs warnings) and add new code(introduce some valid findbugs). Then test-patch may not make difference due to this OK count. Instead, we can add findbugs-exclude file right? this may point the exact place of the findbug to ignore. What do you say? > Findbugs check in test-patch.sh always fails > > > Key: HBASE-5597 > URL: https://issues.apache.org/jira/browse/HBASE-5597 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: David S. Wang >Assignee: David S. Wang > Attachments: HBASE-5597.patch > > > "Fix" is to bump up OK findbugs count in test-patch.properties. Agreed to on > the list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231816#comment-13231816
]
Lars Hofhansl commented on HBASE-5569:
--
Interestingly I saw this right before:
{quote}
2012-03-16 19:24:30,523 DEBUG [Thread-46] regionserver.StoreScanner(499): Stores
canner.peek() is changed where before = rowA/colfamily11:qual1/2561/DeleteColumn
/vlen=0,and after = rowA/colfamily11:qual1/2561/DeleteColumn/vlen=0
{quote}
Which makes no sense, because before and after are the same KV.
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231815#comment-13231815
]
Lars Hofhansl commented on HBASE-5569:
--
Ran testRowMutationMultiThreads another 1000 times on my work machine without
any failures.
Then I ran it at home (much slower machine - but fast SSD) and saw a failure
indeed pretty quickly. Hmm...
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5568) Multi concurrent flushcache() for one region could cause data loss
[ https://issues.apache.org/jira/browse/HBASE-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231804#comment-13231804 ] Hadoop QA commented on HBASE-5568: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518769/HBASE-5568-92v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1215//console This message is automatically generated. > Multi concurrent flushcache() for one region could cause data loss > -- > > Key: HBASE-5568 > URL: https://issues.apache.org/jira/browse/HBASE-5568 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5568-90.patch, HBASE-5568-92v2.patch, > HBASE-5568.patch, HBASE-5568.patch, HBASE-5568v2.patch > > > We could call HRegion#flushcache() concurrently now through > HRegionServer#splitRegion or HRegionServer#flushRegion by HBaseAdmin. > However, we find if HRegion#internalFlushcache() is called concurrently by > multi thread, HRegion.memstoreSize will be calculated wrong. > At the end of HRegion#internalFlushcache(), we will do > this.addAndGetGlobalMemstoreSize(-flushsize), but the flushsize may not the > actual memsize which flushed to hdfs. It cause HRegion.memstoreSize is > negative and prevent next flush if we close this region. > Logs in RS for region e9d827913a056e696c39bc569ea3 > 2012-03-11 16:31:36,690 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 128.0m > 2012-03-11 16:31:37,999 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/8162481165586107427, entries=153106, sequenceid=619316544, > memsize=59.6m, filesize=31.2m > 2012-03-11 16:31:38,830 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 134.8m > 2012-03-11 16:31:39,458 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/3425971951499794221, entries=230183, sequenceid=619316544, > memsize=68.5m, filesize=26.6m > 2012-03-11 16:31:39,459 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~128.1m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 2769ms, sequenceid=619316544, compaction > requested=false > 2012-03-11 16:31:39,459 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 6.8m > 2012-03-11 16:31:39,529 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/1811012969998104626, entries=8002, sequenceid=619332759, > memsize=3.1m, filesize=1.6m > 2012-03-11 16:31:39,640 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/770333473623552048, entries=12231, sequenceid=619332759, > memsize=3.6m, filesize=1.4m > 2012-03-11 16:31:39,641 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~134.8m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 811ms, sequenceid=619332759, compaction > requested=true > 2012-03-11 16:31:39,707 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/5656568849587368557, entries=119, sequenceid=619332979, > memsize=47.4k, filesize=25.6k > 2012-03-11 16:31:39,775 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/794343845650987521, entries=157, sequenceid=619332979, > memsize=47.8k, filesize=19.3k > 2012-03-11 16:31:39,777 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstor
[jira] [Updated] (HBASE-5568) Multi concurrent flushcache() for one region could cause data loss
[ https://issues.apache.org/jira/browse/HBASE-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-5568: Attachment: HBASE-5568-92v2.patch patch for 92 version > Multi concurrent flushcache() for one region could cause data loss > -- > > Key: HBASE-5568 > URL: https://issues.apache.org/jira/browse/HBASE-5568 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5568-90.patch, HBASE-5568-92v2.patch, > HBASE-5568.patch, HBASE-5568.patch, HBASE-5568v2.patch > > > We could call HRegion#flushcache() concurrently now through > HRegionServer#splitRegion or HRegionServer#flushRegion by HBaseAdmin. > However, we find if HRegion#internalFlushcache() is called concurrently by > multi thread, HRegion.memstoreSize will be calculated wrong. > At the end of HRegion#internalFlushcache(), we will do > this.addAndGetGlobalMemstoreSize(-flushsize), but the flushsize may not the > actual memsize which flushed to hdfs. It cause HRegion.memstoreSize is > negative and prevent next flush if we close this region. > Logs in RS for region e9d827913a056e696c39bc569ea3 > 2012-03-11 16:31:36,690 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 128.0m > 2012-03-11 16:31:37,999 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/8162481165586107427, entries=153106, sequenceid=619316544, > memsize=59.6m, filesize=31.2m > 2012-03-11 16:31:38,830 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 134.8m > 2012-03-11 16:31:39,458 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/3425971951499794221, entries=230183, sequenceid=619316544, > memsize=68.5m, filesize=26.6m > 2012-03-11 16:31:39,459 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~128.1m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 2769ms, sequenceid=619316544, compaction > requested=false > 2012-03-11 16:31:39,459 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 6.8m > 2012-03-11 16:31:39,529 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/1811012969998104626, entries=8002, sequenceid=619332759, > memsize=3.1m, filesize=1.6m > 2012-03-11 16:31:39,640 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/770333473623552048, entries=12231, sequenceid=619332759, > memsize=3.6m, filesize=1.4m > 2012-03-11 16:31:39,641 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~134.8m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 811ms, sequenceid=619332759, compaction > requested=true > 2012-03-11 16:31:39,707 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/5656568849587368557, entries=119, sequenceid=619332979, > memsize=47.4k, filesize=25.6k > 2012-03-11 16:31:39,775 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/794343845650987521, entries=157, sequenceid=619332979, > memsize=47.8k, filesize=19.3k > 2012-03-11 16:31:39,777 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~6.8m for region > writetest1,,1331454657410.e9d827913a05 > 6e696c39bc569ea3f99f. in 318ms, sequenceid=619332979, compaction > requested=true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231795#comment-13231795 ] Hadoop QA commented on HBASE-5521: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518757/HBASE-5521.D2097.8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 165 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestImportTsv Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1214//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1214//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1214//console This message is automatically generated. > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch, > HBASE-5521.D2097.8.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231788#comment-13231788 ] Phabricator commented on HBASE-5521: tedyu has commented on the revision "HBASE-5521 [jira] Move compression/decompression to an encoder specific encoding context". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java:202 How about naming this method encodeData() ? src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:106 Typo: 'to start' src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:109 I think this method should be in the interface. src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockEncodingContext.java:34 When getOnDiskBytesWithHeader() is called, encoding is done already, right ? src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:131 I think this method should be in the interface. src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:177 Is this method called anywhere ? src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:56 Does 'as well as' mean 'or' here ? src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:61 I still think this method should be renamed because compression is optional. REVISION DETAIL https://reviews.facebook.net/D2097 > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch, > HBASE-5521.D2097.8.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5597) Findbugs check in test-patch.sh always fails
[ https://issues.apache.org/jira/browse/HBASE-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231774#comment-13231774 ] Hadoop QA commented on HBASE-5597: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518754/HBASE-5597.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 161 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestImportTsv Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1213//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1213//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1213//console This message is automatically generated. > Findbugs check in test-patch.sh always fails > > > Key: HBASE-5597 > URL: https://issues.apache.org/jira/browse/HBASE-5597 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: David S. Wang >Assignee: David S. Wang > Attachments: HBASE-5597.patch > > > "Fix" is to bump up OK findbugs count in test-patch.properties. Agreed to on > the list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5596) Few minor bugs from HBASE-5209
[ https://issues.apache.org/jira/browse/HBASE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231775#comment-13231775 ] Hadoop QA commented on HBASE-5596: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518752/HBASE_5596.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 163 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestMasterShutdown org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit org.apache.hadoop.hbase.client.TestInstantSchemaChangeFailover org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1212//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1212//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1212//console This message is automatically generated. > Few minor bugs from HBASE-5209 > -- > > Key: HBASE-5596 > URL: https://issues.apache.org/jira/browse/HBASE-5596 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.92.1, 0.94.0 >Reporter: David S. Wang >Assignee: David S. Wang >Priority: Minor > Attachments: HBASE_5596.patch > > > A few leftover bugs from HBASE-5209. Comments are documented here: > https://reviews.apache.org/r/3892/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5521: --- Attachment: HBASE-5521.D2097.8.patch heyongqiang updated the revision "HBASE-5521 [jira] Move compression/decompression to an encoder specific encoding context". Reviewers: JIRA, dhruba, tedyu, sc, mbautin address mbautin & tedyu's comments. REVISION DETAIL https://reviews.facebook.net/D2097 AFFECTED FILES src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileDataBlockEncoder.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDecodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultDecodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockEncodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch, > HBASE-5521.D2097.8.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo should compare regionId as well
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231757#comment-13231757 ] Hudson commented on HBASE-5563: --- Integrated in HBase-0.92 #325 (See [https://builds.apache.org/job/HBase-0.92/325/]) HBASE-5563 Addendum: Updating CHANGES.txt (Revision 1301782) HBASE-5563 Add comparison of regionId to HRegionInfo#compareTo (chunhui and jmhsieh) (Revision 1301780) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt jmhsieh : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java > HRegionInfo#compareTo should compare regionId as well > - > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5597) Findbugs check in test-patch.sh always fails
[ https://issues.apache.org/jira/browse/HBASE-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5597: - Status: Patch Available (was: Open) > Findbugs check in test-patch.sh always fails > > > Key: HBASE-5597 > URL: https://issues.apache.org/jira/browse/HBASE-5597 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: David S. Wang >Assignee: David S. Wang > Attachments: HBASE-5597.patch > > > "Fix" is to bump up OK findbugs count in test-patch.properties. Agreed to on > the list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5597) Findbugs check in test-patch.sh always fails
Findbugs check in test-patch.sh always fails Key: HBASE-5597 URL: https://issues.apache.org/jira/browse/HBASE-5597 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang "Fix" is to bump up OK findbugs count in test-patch.properties. Agreed to on the list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5597) Findbugs check in test-patch.sh always fails
[ https://issues.apache.org/jira/browse/HBASE-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang reassigned HBASE-5597: Assignee: David S. Wang > Findbugs check in test-patch.sh always fails > > > Key: HBASE-5597 > URL: https://issues.apache.org/jira/browse/HBASE-5597 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: David S. Wang >Assignee: David S. Wang > > "Fix" is to bump up OK findbugs count in test-patch.properties. Agreed to on > the list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5597) Findbugs check in test-patch.sh always fails
[ https://issues.apache.org/jira/browse/HBASE-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5597: - Attachment: HBASE-5597.patch > Findbugs check in test-patch.sh always fails > > > Key: HBASE-5597 > URL: https://issues.apache.org/jira/browse/HBASE-5597 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: David S. Wang >Assignee: David S. Wang > Attachments: HBASE-5597.patch > > > "Fix" is to bump up OK findbugs count in test-patch.properties. Agreed to on > the list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5596) Few minor bugs from HBASE-5209
[ https://issues.apache.org/jira/browse/HBASE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5596: - Attachment: HBASE_5596.patch > Few minor bugs from HBASE-5209 > -- > > Key: HBASE-5596 > URL: https://issues.apache.org/jira/browse/HBASE-5596 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.92.1, 0.94.0 >Reporter: David S. Wang >Assignee: David S. Wang >Priority: Minor > Attachments: HBASE_5596.patch > > > A few leftover bugs from HBASE-5209. Comments are documented here: > https://reviews.apache.org/r/3892/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5596) Few minor bugs from HBASE-5209
[ https://issues.apache.org/jira/browse/HBASE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5596: - Status: Patch Available (was: Open) > Few minor bugs from HBASE-5209 > -- > > Key: HBASE-5596 > URL: https://issues.apache.org/jira/browse/HBASE-5596 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.92.1, 0.94.0 >Reporter: David S. Wang >Assignee: David S. Wang >Priority: Minor > Attachments: HBASE_5596.patch > > > A few leftover bugs from HBASE-5209. Comments are documented here: > https://reviews.apache.org/r/3892/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231741#comment-13231741
]
Zhihong Yu commented on HBASE-5592:
---
This is an improvement, right ?
Should it go into 0.92 which becomes stable release ?
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5593) Reverse DNS resolution in regionServerStartup() does not strip trailing dot
[ https://issues.apache.org/jira/browse/HBASE-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231738#comment-13231738 ] Zhihong Yu commented on HBASE-5593: --- Yes. > Reverse DNS resolution in regionServerStartup() does not strip trailing dot > --- > > Key: HBASE-5593 > URL: https://issues.apache.org/jira/browse/HBASE-5593 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.6 >Reporter: David S. Wang >Assignee: David S. Wang > Fix For: 0.90.7 > > Attachments: HBASE-5593.patch > > > HBASE-4109 covered the removal of trailing dots in PTR records from reverse > DNS lookups. We seem to have missed a case in HMaster#regionServerStartup(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo should compare regionId as well
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5563: -- Summary: HRegionInfo#compareTo should compare regionId as well (was: HRegionInfo#compareTo add the comparison of regionId) > HRegionInfo#compareTo should compare regionId as well > - > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5533) Add more metrics to HBase
[ https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231733#comment-13231733 ] stack commented on HBASE-5533: -- Those metrics look beautiful. NamedThreadFactory overlaps with functionality in the (badly named) util.Threads class. You might check it out to make sure we're not duplicating facility. No need of these' + * Copyright 2010 The Apache Software Foundation' lines when you are adding headers. TestMetricsHistogram is missing license header. Use something like http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/util/StringUtils.html#limitDecimalTo2%28double%29 when you are in RegionServerMetrics#toString outputting your new values else they could go ugly with lots of decimal places (see your screen shot). Fix the minors above and lets get it in. Good stuff Shaneal. > Add more metrics to HBase > - > > Key: HBASE-5533 > URL: https://issues.apache.org/jira/browse/HBASE-5533 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.94.0 >Reporter: Shaneal Manek >Assignee: Shaneal Manek >Priority: Minor > Attachments: BlockingQueueContention.java, HBASE-5533-0.92-v4.patch, > TimingOverhead.java, hbase-5533-0.92.patch, hbase5533-0.92-v2.patch, > hbase5533-0.92-v3.patch, histogram_web_ui.png > > > To debub/monitor production clusters, there are some more metrics I wish I > had available. > In particular: > - Although the average FS latencies are useful, a 'histogram' of recent > latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc) > would be more useful > - Similar histograms of latencies on common operations (GET, PUT, DELETE) > would be useful > - Counting the number of accesses to each region to detect hotspotting > - Exposing the current number of HLog files -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231732#comment-13231732 ] Phabricator commented on HBASE-5521: tedyu has commented on the revision "HBASE-5521 [jira] Move compression/decompression to an encoder specific encoding context". Good progress. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java:330 Extra empty line. src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:55 Can 'contain returning results' be explained a little more ? src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1624 'to seek back' can be replaced with 'backward' src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:38 'compress' -> 'compresses' src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:60 Since compression is optional, should this method be named encodeKeyValuesFollowedByOptionalCompression :-) src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:122 Would createDataBlockEncodingContext be a better name ? src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:127 'an encoder' src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java:200 Are this javadoc and method name consistent with what line 212 returns ? src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:64 Please add javadoc for the parameters src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:133 Add headerBytes to @param src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java:208 'decoding' -> 'compression' src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:156 Please add @Override for methods in the interface. REVISION DETAIL https://reviews.facebook.net/D2097 > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231729#comment-13231729 ] Hudson commented on HBASE-5563: --- Integrated in HBase-0.94 #37 (See [https://builds.apache.org/job/HBase-0.94/37/]) HBASE-5563 Add comparison of regionId to HRegionInfo#compareTo (chunhui and jmhsieh) (Revision 1301783) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java > HRegionInfo#compareTo add the comparison of regionId > > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5589) Add of the offline call to the Master Interface
[ https://issues.apache.org/jira/browse/HBASE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231726#comment-13231726 ] Jonathan Hsieh commented on HBASE-5589: --- Need to check to see if order matters for rolling upgrade. > Add of the offline call to the Master Interface > --- > > Key: HBASE-5589 > URL: https://issues.apache.org/jira/browse/HBASE-5589 > Project: HBase > Issue Type: Sub-task > Components: hbck >Affects Versions: 0.90.6, 0.92.0, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > Hbck from HBASE-5128 requires an offline method on the master to properly > cleanup state during certain assignment repair operations. This will this > method will be added to recent and older versions of HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231725#comment-13231725 ] Phabricator commented on HBASE-5335: stack has commented on the revision "[jira] [HBASE-5335] Dynamic Schema Config". A few minors. Looks good. Will need fat release note when committed. Lars, might go into 0.94? INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java:776 This looks nice in the output? src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java:582 Seems like some of this method could be unified w/ the HCD#getValues (as per Ted comment) src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:947 What is the dedupe problem? Dedup of HTD properties? src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:938 Does this have to be public? Only used inside this package? src/main/java/org/apache/hadoop/hbase/util/CompoundConfiguration.java:1 Configuration in hadoop is in the conf package. Should we have one in hbase. We could put this there. Or, earlier I suggest it be an inner class of HBaseConfiguration or just a feature of HBC... you ignored that comment 'cos it silly? src/main/ruby/hbase/admin.rb:187 How does this work in shell? I can't picture it looking at code. Looks like user says ADVANCED and then passes k/vs? If so, how you think we going to explain notion of ADVANCED? src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java:169 Hmm... above I suggest this be a package private method but see here it needs to be public... at least for this test. REVISION DETAIL https://reviews.facebook.net/D2247 > Dynamic Schema Configurations > - > > Key: HBASE-5335 > URL: https://issues.apache.org/jira/browse/HBASE-5335 > Project: HBase > Issue Type: New Feature >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg > Labels: configuration, schema > Attachments: D2247.1.patch, D2247.2.patch > > > Currently, the ability for a core developer to add per-table & per-CF > configuration settings is very heavyweight. You need to add a reserved > keyword all the way up the stack & you have to support this variable > long-term if you're going to expose it explicitly to the user. This has > ended up with using Configuration.get() a lot because it is lightweight and > you can tweak settings while you're trying to understand system behavior > [since there are many config params that may never need to be tuned]. We > need to add the ability to put & read arbitrary KV settings in the HBase > schema. Combined with online schema change, this will allow us to safely > iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5522) hbase 0.92 test artifacts are missing from Maven central
[ https://issues.apache.org/jira/browse/HBASE-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231723#comment-13231723 ] Alan Gates commented on HBASE-5522: --- My last comment can be ignored. It turned out to be a Hive ivy configuration issue. > hbase 0.92 test artifacts are missing from Maven central > > > Key: HBASE-5522 > URL: https://issues.apache.org/jira/browse/HBASE-5522 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.92.0 >Reporter: Roman Shaposhnik >Assignee: stack > Fix For: 0.92.1, 0.94.0 > > Attachments: 5522.txt > > > Could someone with enough karma, please, publish the test artifacts for > 0.92.0? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231717#comment-13231717 ] Hadoop QA commented on HBASE-5521: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518742/HBASE-5521.D2097.7.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 165 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.util.TestHBaseFsck Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1211//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1211//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1211//console This message is automatically generated. > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5596) Few minor bugs from HBASE-5209
Few minor bugs from HBASE-5209 -- Key: HBASE-5596 URL: https://issues.apache.org/jira/browse/HBASE-5596 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.1, 0.94.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor A few leftover bugs from HBASE-5209. Comments are documented here: https://reviews.apache.org/r/3892/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231715#comment-13231715 ] Phabricator commented on HBASE-5521: mbautin has commented on the revision "HBASE-5521 [jira] Move compression/decompression to an encoder specific encoding context". Yongqiang: here are some more comments, mostly very minor, but a couple of real ones, too. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:38 It -> it src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java:79 Remove unnecessary concatenation src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:644 Add an empty line here src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:829 Code style: add space after (int) src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:952-955 Could the close operation fail? If so, would it make sense to attempt the second close operation in a finally block? src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1160 Nice! +1 on making more fields final. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1445-1446 Can these be made private? src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1729 Code style: space between if and ( I will add a lint rule to catch this, too. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1742-1745 nit: in multi-line comments like this, please capitalize the first word and end sentences with a full stop. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java:94 [nit] create -> Create src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java:111 nit: specify what this returns, or remove @return src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java:42-45 Do we have any subclasses of HFileDataBlockEncoderImpl, or are these fields accessed from other classes in the package? If latter, please change access to package-private. This is a more restrictive access level than protected, because classes from the same package can actually access protected members (I was surprised when I learned about this). src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java:86 nit: use a // comment src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java:263-276 Here is probably one of the few important comments in my current batch. There is code duplication between these two methods. Can we merge these two methods in the HFile encoder interface into one, and get rid of this code duplication in the implementation? src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java:41 Why does this have to be protected? This class is a singleton and we should not create it anywhere else. Instead, use NoOpDataBlockEncoder.INSTANCE. src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java:358 Is DataBlockEncoding.getAllEncoders() used anywhere after you have removed this line? If not, I think we should get rid of it since it is confusing to have a function called getAllEncoders() that does not actually return all encoders. src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java:20 This file contains a lot of code duplicated from TestHFileBlock. Dhruba's intent, as I understood it, to have a "frozen" implementation without HBase-level checksums that tests would use to ensure compatibility before and after the checksum patch. Therefore, we should probably minimize changes to this file and emphasize that in comments explicitly and conspicuously. Please work with Dhruba to address the TestHFileBlockCompatibility issues. REVISION DETAIL https://reviews.facebook.net/D2097 > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-co
[jira] [Commented] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup
[ https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231712#comment-13231712 ] jirapos...@reviews.apache.org commented on HBASE-5209: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3892/#review6055 --- Ship it! Dave, Additions look good to me. Since this is a little while since the others were committed please file another jira, and then I'll commit. Thanks! - jmhsieh On 2012-03-15 13:16:01, David Wang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3892/ bq. --- bq. bq. (Updated 2012-03-15 13:16:01) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Problem: bq. There is no method in the HBase client-facing APIs to determine which of the masters is currently active. This can be especially useful in setups with multiple backup masters. bq. bq. Solution: bq. Augment ClusterStatus to return the currently active master and the list of backup masters. bq. bq. Notes: bq. * I uncovered a race condition in ActiveMasterManager, between when it determines that it did not win the original race to be the active master, and when it reads the ServerName of the active master. If the active master goes down in that time, the read to determine the active master's ServerName will fail ungracefully and the candidate master will abort. The solution incorporated in this patch is to check to see if the read of the ServerName succeeded before trying to use it. bq. * I fixed some minor formatting issues while going through the code. I can take these changes out if it is considered improper to commit such non-related changes with the main changes. bq. bq. bq. This addresses bug HBASE-5209. bq. https://issues.apache.org/jira/browse/HBASE-5209 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 9df4c10 bq.src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 7416ae2 bq.src/main/java/org/apache/hadoop/hbase/master/HMaster.java e2bbbd0 bq. bq. Diff: https://reviews.apache.org/r/3892/diff bq. bq. bq. Testing bq. --- bq. bq. * Ran mvn -P localTests test multiple times - no new tests fail bq. * Ran mvn -P localTests -Dtest=TestActiveMasterManager test multiple runs - no failures bq. * Ran mvn -P localTests -Dtest=TestMasterFailover test multiple runs - no failures bq. * Started active and multiple backup masters, then killed active master, then brought it back up (will now be a backup master) bq.* Did the following before and after killing bq. * hbase hbck -details - checked output to see that active and backup masters are reported properly bq. * zk_dump - checked that active and backup masters are reported properly bq. * Started cluster with no backup masters to make sure change operates correctly that way bq. * Tested build with this diff vs. build without this diff, in all combinations of client and server bq.* Verified that new client can run against old servers without incident and with the defaults applied. bq.* Note that old clients get an error when running against new servers, because the old readFields() code in ClusterStatus does not handle exceptions of any kind. This is not solvable, at least in the scope of this change. bq. bq. 12/02/15 15:15:38 INFO zookeeper.ClientCnxn: Session establishment complete on server haus02.sf.cloudera.com/172.29.5.33:30181, sessionid = 0x135834c75e20008, negotiated timeout = 5000 bq. 12/02/15 15:15:39 ERROR io.HbaseObjectWritable: Error in readFields bq. A record version mismatch occured. Expecting v2, found v3 bq. at org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46) bq. at org.apache.hadoop.hbase.ClusterStatus.readFields(ClusterStatus.java:247) bq. at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:583) bq. at org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297) bq. bq. * Ran dev-support/test-patch.sh - no new issues fail: bq. bq. -1 overall. bq. bq. +1 @author. The patch does not contain any @author tags. bq. bq. +1 tests included. The patch appears to include 7 new or modified tests. bq. bq. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. bq. bq. +1 javac. The applied patch does not increase the total number of javac compiler warnings. bq. bq. +1 findbugs. The pat
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[
https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231705#comment-13231705
]
Zhihong Yu commented on HBASE-5521:
---
I have verified that failed tests pass now.
There're a lot of whitespace warnings from lint:
{code}
>>> Lint for
>>> src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileDataBlockEncoder.java:
Error (TXT6) Trailing Whitespace
This line contains trailing whitespace.
{code}
Please give me a few moment in going over the patch one more time.
> Move compression/decompression to an encoder specific encoding context
> --
>
> Key: HBASE-5521
> URL: https://issues.apache.org/jira/browse/HBASE-5521
> Project: HBase
> Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch,
> HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch,
> HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch
>
>
> As part of working on HBASE-5313, we want to add a new columnar
> encoder/decoder. It makes sense to move compression to be part of
> encoder/decoder:
> 1) a scanner for a columnar encoded block can do lazy decompression to a
> specific part of a key value object
> 2) avoid an extra bytes copy from encoder to hblock-writer.
> If there is no encoder specified for a writer, the HBlock.Writer will use a
> default compression-context to do something very similar to today's code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231707#comment-13231707
]
Jesse Yates commented on HBASE-5592:
Oh, btw my issue with this patch is that it requires a knowledge much deeper
than previously required to get a reference to the table and breaks the current
paradigm of using the simple, one-word commands. Also, the table obtained this
way isn't going to have any of the nice formatting we are all used to (a little
superfluous, but nice smile).
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231704#comment-13231704
]
Jesse Yates commented on HBASE-5592:
They aren't going to interfere, just don't see the point of having Ben's when
5548 goes in. I can revert his change when doing the commit on mine, but that
seems a bit roundabout to me ;)
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231700#comment-13231700
]
stack commented on HBASE-5592:
--
I can revert. Would be interested in Bens' opinion first.
What if we left this in 0.92 and did your version in 0.94+ Jesse? Do they
interfere?
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5593) Reverse DNS resolution in regionServerStartup() does not strip trailing dot
[ https://issues.apache.org/jira/browse/HBASE-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231698#comment-13231698 ] stack commented on HBASE-5593: -- That is given directly to Strings.domainNamePointerToHostName so you would like a test that shows that if a dot suffix, domainNamePointerToHostName will drop it? That should be enough? > Reverse DNS resolution in regionServerStartup() does not strip trailing dot > --- > > Key: HBASE-5593 > URL: https://issues.apache.org/jira/browse/HBASE-5593 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.6 >Reporter: David S. Wang >Assignee: David S. Wang > Fix For: 0.90.7 > > Attachments: HBASE-5593.patch > > > HBASE-4109 covered the removal of trailing dots in PTR records from reverse > DNS lookups. We seem to have missed a case in HMaster#regionServerStartup(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5575) Configure Arcanist lint engine for HBase
[ https://issues.apache.org/jira/browse/HBASE-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231697#comment-13231697 ] Hudson commented on HBASE-5575: --- Integrated in HBase-TRUNK #2686 (See [https://builds.apache.org/job/HBase-TRUNK/2686/]) [jira] [HBASE-5575] Configure Arcanist lint engine for HBase Summary: Adding Java lint engine to HBase's .arcconfig. The lint engine itself is part of the arc-jira repository that lives at https://github.com/facebook/arc-jira/. There are some changes to be made there to prevent lint from reporting errors for unmodified lines. Test Plan: arc lint Reviewers: nspiegelberg, Kannan, Karthik, JIRA, Liyin Reviewed By: Liyin Differential Revision: https://reviews.facebook.net/D2289 (Revision 1301751) Result = SUCCESS mbautin : Files : * /hbase/trunk/.arcconfig > Configure Arcanist lint engine for HBase > > > Key: HBASE-5575 > URL: https://issues.apache.org/jira/browse/HBASE-5575 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: Enabling-lint-2012-03-16_13_40_37.patch > > > We need to enable Arcanist lint engine in HBase, so that a commit could be > checked by running "arc lint". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231696#comment-13231696 ] Hudson commented on HBASE-5563: --- Integrated in HBase-TRUNK #2686 (See [https://builds.apache.org/job/HBase-TRUNK/2686/]) HBASE-5563 Add comparison of regionId to HRegionInfo#compareTo (chunhui and jmhsieh) (Revision 1301779) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java > HRegionInfo#compareTo add the comparison of regionId > > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231694#comment-13231694
]
Hudson commented on HBASE-5592:
---
Integrated in HBase-TRUNK #2686 (See
[https://builds.apache.org/job/HBase-TRUNK/2686/])
HBASE-5592 Make it easier to get a table from shell (Revision 1301715)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/ruby/hbase/table.rb
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5572) KeeperException.SessionExpiredException management could be improved in Master
[
https://issues.apache.org/jira/browse/HBASE-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231693#comment-13231693
]
Hudson commented on HBASE-5572:
---
Integrated in HBase-TRUNK #2686 (See
[https://builds.apache.org/job/HBase-TRUNK/2686/])
HBASE-5549 HBASE-5572 Master can fail if ZooKeeper session expires (N
Keywal) (Revision 1301775)
Result = SUCCESS
tedyu :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterZKSessionRecovery.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java
> KeeperException.SessionExpiredException management could be improved in Master
> --
>
> Key: HBASE-5572
> URL: https://issues.apache.org/jira/browse/HBASE-5572
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5572.v1.patch, 5572.v2.patch, 5572.v2.patch,
> 5572.v2.patch
>
>
> Synthesis:
> 1) TestMasterZKSessionRecovery distinguish two cases on
> SessionExpiredException. One is explicitly not managed. However, is seems
> that there is no reason for this.
> 2) The issue lies in ActiveMasterManager#blockUntilBecomingActiveMaster, a
> quite complex function, with a useless recursive call.
> 3) TestMasterZKSessionRecovery#testMasterZKSessionRecoverySuccess is
> equivalent to TestZooKeeper#testMasterSessionExpired
> 4) TestMasterZKSessionRecovery#testMasterZKSessionRecoveryFailure can be
> removed if we merge the two cases mentioned above.
> Changes are:
> 2) Changing ActiveMasterManager#blockUntilBecomingActiveMaster to have a
> single case and remove recursion
> 1) Removing TestMasterZKSessionRecovery
> Detailed justification:
> testMasterZKSessionRecoveryFailure says:
> {noformat}
> /**
>* Negative test of master recovery from zk session expiry.
>*
>* Starts with one master. Fakes the master zk session expired.
>* Ensures the master cannot recover the expired zk session since
>* the master zk node is still there.
>*/
> public void testMasterZKSessionRecoveryFailure() throws Exception {
> MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
> HMaster m = cluster.getMaster();
> m.abort("Test recovery from zk session expired",
> new KeeperException.SessionExpiredException());
> assertTrue(m.isStopped());
> }
> {noformat}
> This tests works, i.e. the assertion is always verified.
> But do we really want this behavior?
> When looking at the code, we see that this what's happening is strange:
> - HMaster#abort calls Master#abortNow. If HMaster#abortNow returns false
> HMaster#abort stops the master.
> - HMaster#abortNow checks the exception type. As it's a
> SessionExpiredException it will try to recover, calling
> HMaster#tryRecoveringExpiredZKSession. If it cannot, it will return false
> (and that will make HMaster#abort stopping the master)
> - HMaster#tryRecoveringExpiredZKSession recreates a ZooKeeperConnection and
> then try to become the active master. If it cannot, it will return false (and
> that will make HMaster#abort stopping the master).
> - HMaster#becomeActiveMaster returns the result of
> ActiveMasterManager#blockUntilBecomingActiveMaster.
> blockUntilBecomingActiveMaster says it will return false if there is any
> error preventing it to become the active master.
> - ActiveMasterManager#blockUntilBecomingActiveMaster reads ZK for the master
> address. If it's the same port & host, it deletes the nodes, that will start
> a recursive call to blockUntilBecomingActiveMaster. This second call succeeds
> (we became the active master) and return true. This result is ignored by the
> first blockUntilBecomingActiveMaster: it return false (even if we actually
> became the active master), hence the whole suite call returns false and
> HMaster#abort stops the master.
> In other words, the co
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231695#comment-13231695 ] Hudson commented on HBASE-5549: --- Integrated in HBase-TRUNK #2686 (See [https://builds.apache.org/job/HBase-TRUNK/2686/]) HBASE-5549 HBASE-5572 Master can fail if ZooKeeper session expires (N Keywal) (Revision 1301775) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterZKSessionRecovery.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231692#comment-13231692 ] Mikhail Bautin commented on HBASE-5521: --- +1 I have sucessfully re-run tests that failed on Jenkins locally: Running org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.904 sec Running org.apache.hadoop.hbase.io.hfile.TestReseekTo Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.846 sec Running org.apache.hadoop.hbase.io.hfile.TestHFile Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.11 sec Running org.apache.hadoop.hbase.coprocessor.TestProcessRowEndpoint Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.273 sec Results : Tests run: 10, Failures: 0, Errors: 0, Skipped: 0 @Committers: this patch has been out for a while, and Yongqiang is addressing final minor comments. Would anyone else +1 this? > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231680#comment-13231680
]
Lars Hofhansl commented on HBASE-5569:
--
Hmm... I ran the tests (all of them - including testRowMutationMultiThreads)
over 4000 times, didn't fail.
testMultiRowMutationMultiThreads is definitely fixed (failed after a few dozen
executions before).
There might be yet another much rarer problem with testRowMutationMultiThreads.
I've never seen it fail on the build machines, yet.
Any chance you could attach the latest logs (as zip or tar)?
Btw, this:
{code}
2012-03-14 03:14:02,146 DEBUG [Thread-51]
regionserver.TestAtomicOperation$1(305): keyvalues=NONE
Exception in thread "Thread-51"
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.fail(Assert.java:56)
at
org.apache.hadoop.hbase.regionserver.TestAtomicOperation$1.run(TestAtomicOperation.java:307)
2012-03-14 03:14:02,228 DEBUG [Thread-92]
regionserver.TestAtomicOperation$1(279): flushing
{code}
Is just when the test detects the problem. The actual problem should be in the
logs some time before that.
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5581) Creating a table with invalid syntax does not give an error message when it fails
[
https://issues.apache.org/jira/browse/HBASE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231678#comment-13231678
]
Phabricator commented on HBASE-5581:
mbautin has committed the revision "[jira] [HBASE-5581] [89-fb] Creating a
table with invalid syntax does not give an error message when it fails".
REVISION DETAIL
https://reviews.facebook.net/D2343
COMMIT
https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1301789
> Creating a table with invalid syntax does not give an error message when it
> fails
> -
>
> Key: HBASE-5581
> URL: https://issues.apache.org/jira/browse/HBASE-5581
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Binu John
>Priority: Minor
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5581trunk.patch, D2343.1.patch
>
>
> Creating a table with invalid syntax does not give an error message when it
> fails. In this case, it doesn't actually create the CF requested, but doesn't
> give any indication to the user that it failed.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true, NUMREGIONS
> => 20, SPLITALGO => "HexStringSplit", COMPRESSION => 'LZO', BLOOMFILTER =>
> 'ROW'}
> 0 row(s) in 3.0930 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => []}
> true
> 1 row(s) in 0.1430 seconds
>
> Putting {NUMREGIONS => 20, SPLITALGO => "HexStringSplit"} into a separate
> stanza works fine, so the feature is fine.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true,
> COMPRESSION => 'LZO', BLOOMFILTER => 'ROW'}, {NUMREGIONS => 20, SPLITALGO =>
> "HexStringSplit"}
> 0 row(s) in 2.7860 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => [{NAME => 'test', DATA_BLOCK_ENCODING =>
> 'NONE', true
> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'LZO',
> VERSIONS
> => '1', TTL => '2147483647', BLOCKSIZE => '65536', BLOOMFILTER_ERRORRATE =>
> '
> 0.01', ENCODE_ON_DISK => 'true', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}]}
>
> We should throw an error if we can't create the CF so it's clear to the user.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231676#comment-13231676 ] Lars Hofhansl commented on HBASE-5549: -- Hey N., is this dependent on HBASE-5399. If not, it might be a good candidate for 0.94. > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231674#comment-13231674
]
Hudson commented on HBASE-5592:
---
Integrated in HBase-0.92 #324 (See
[https://builds.apache.org/job/HBase-0.92/324/])
HBASE-5592 Make it easier to get a table from shell (Revision 1301717)
Result = FAILURE
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/ruby/hbase/table.rb
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231671#comment-13231671 ] Phabricator commented on HBASE-5335: tedyu has commented on the revision "[jira] [HBASE-5335] Dynamic Schema Config". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/util/CompoundConfiguration.java:65 This seems to be the only place com.google.common.collect.Lists is used. Can we instantiate ArrayList directly ? src/main/java/org/apache/hadoop/hbase/util/CompoundConfiguration.java:120 This can be brought onto line 119. src/main/java/org/apache/hadoop/hbase/util/CompoundConfiguration.java:143 Should getRaw() be called here ? src/main/java/org/apache/hadoop/hbase/util/CompoundConfiguration.java:153 This can be removed. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java:526 Can this be unified with HColumnDescriptor.getValues() ? REVISION DETAIL https://reviews.facebook.net/D2247 > Dynamic Schema Configurations > - > > Key: HBASE-5335 > URL: https://issues.apache.org/jira/browse/HBASE-5335 > Project: HBase > Issue Type: New Feature >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg > Labels: configuration, schema > Attachments: D2247.1.patch, D2247.2.patch > > > Currently, the ability for a core developer to add per-table & per-CF > configuration settings is very heavyweight. You need to add a reserved > keyword all the way up the stack & you have to support this variable > long-term if you're going to expose it explicitly to the user. This has > ended up with using Configuration.get() a lot because it is lightweight and > you can tweak settings while you're trying to understand system behavior > [since there are many config params that may never need to be tuned]. We > need to add the ability to put & read arbitrary KV settings in the HBase > schema. Combined with online schema change, this will allow us to safely > iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5521: --- Attachment: HBASE-5521.D2097.7.patch heyongqiang updated the revision "HBASE-5521 [jira] Move compression/decompression to an encoder specific encoding context". Reviewers: JIRA, dhruba, tedyu, sc, mbautin fix tests failures REVISION DETAIL https://reviews.facebook.net/D2097 AFFECTED FILES src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileDataBlockEncoder.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDecodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultDecodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockEncodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch, HBASE-5521.D2097.7.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5563: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.92.2 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > HRegionInfo#compareTo add the comparison of regionId > > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231662#comment-13231662 ] Jonathan Hsieh commented on HBASE-5563: --- Committed to 0.92/0.94/trunk. Thanks for initial patch chunhui, thanks for review stack. > HRegionInfo#compareTo add the comparison of regionId > > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231661#comment-13231661 ] Jonathan Hsieh commented on HBASE-5563: --- @stack @lars -- 0.94 ok too? > HRegionInfo#compareTo add the comparison of regionId > > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5549: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5572) KeeperException.SessionExpiredException management could be improved in Master
[
https://issues.apache.org/jira/browse/HBASE-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5572:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Resolved as part of HBASE-5549
> KeeperException.SessionExpiredException management could be improved in Master
> --
>
> Key: HBASE-5572
> URL: https://issues.apache.org/jira/browse/HBASE-5572
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5572.v1.patch, 5572.v2.patch, 5572.v2.patch,
> 5572.v2.patch
>
>
> Synthesis:
> 1) TestMasterZKSessionRecovery distinguish two cases on
> SessionExpiredException. One is explicitly not managed. However, is seems
> that there is no reason for this.
> 2) The issue lies in ActiveMasterManager#blockUntilBecomingActiveMaster, a
> quite complex function, with a useless recursive call.
> 3) TestMasterZKSessionRecovery#testMasterZKSessionRecoverySuccess is
> equivalent to TestZooKeeper#testMasterSessionExpired
> 4) TestMasterZKSessionRecovery#testMasterZKSessionRecoveryFailure can be
> removed if we merge the two cases mentioned above.
> Changes are:
> 2) Changing ActiveMasterManager#blockUntilBecomingActiveMaster to have a
> single case and remove recursion
> 1) Removing TestMasterZKSessionRecovery
> Detailed justification:
> testMasterZKSessionRecoveryFailure says:
> {noformat}
> /**
>* Negative test of master recovery from zk session expiry.
>*
>* Starts with one master. Fakes the master zk session expired.
>* Ensures the master cannot recover the expired zk session since
>* the master zk node is still there.
>*/
> public void testMasterZKSessionRecoveryFailure() throws Exception {
> MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
> HMaster m = cluster.getMaster();
> m.abort("Test recovery from zk session expired",
> new KeeperException.SessionExpiredException());
> assertTrue(m.isStopped());
> }
> {noformat}
> This tests works, i.e. the assertion is always verified.
> But do we really want this behavior?
> When looking at the code, we see that this what's happening is strange:
> - HMaster#abort calls Master#abortNow. If HMaster#abortNow returns false
> HMaster#abort stops the master.
> - HMaster#abortNow checks the exception type. As it's a
> SessionExpiredException it will try to recover, calling
> HMaster#tryRecoveringExpiredZKSession. If it cannot, it will return false
> (and that will make HMaster#abort stopping the master)
> - HMaster#tryRecoveringExpiredZKSession recreates a ZooKeeperConnection and
> then try to become the active master. If it cannot, it will return false (and
> that will make HMaster#abort stopping the master).
> - HMaster#becomeActiveMaster returns the result of
> ActiveMasterManager#blockUntilBecomingActiveMaster.
> blockUntilBecomingActiveMaster says it will return false if there is any
> error preventing it to become the active master.
> - ActiveMasterManager#blockUntilBecomingActiveMaster reads ZK for the master
> address. If it's the same port & host, it deletes the nodes, that will start
> a recursive call to blockUntilBecomingActiveMaster. This second call succeeds
> (we became the active master) and return true. This result is ignored by the
> first blockUntilBecomingActiveMaster: it return false (even if we actually
> became the active master), hence the whole suite call returns false and
> HMaster#abort stops the master.
> In other words, the comment says "Ensures the master cannot recover the
> expired zk session since the master zk node is still there." but we're
> actually doing a check just for this and deleting the node. If we were not
> ignoring the result, we would return "true", so we would not stop the master,
> so the test would fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5549: -- Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Integrated to TRUNK. Thanks for the patch, N. Thanks for the review, Stack. > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231646#comment-13231646 ] Hadoop QA commented on HBASE-5549: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518730/5549.v11.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 161 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1210//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1210//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1210//console This message is automatically generated. > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
[ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231644#comment-13231644 ] Phabricator commented on HBASE-5521: mbautin has commented on the revision "HBASE-5521 [jira] Move compression/decompression to an encoder specific encoding context". Yongqiang: we now have a linter available in HBase trunk. Could you please run "arc lint", resolve lint warnings, and resubmit the diff with "arc diff --preview"? REVISION DETAIL https://reviews.facebook.net/D2097 > Move compression/decompression to an encoder specific encoding context > -- > > Key: HBASE-5521 > URL: https://issues.apache.org/jira/browse/HBASE-5521 > Project: HBase > Issue Type: Improvement >Reporter: He Yongqiang >Assignee: He Yongqiang > Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, > HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, > HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch > > > As part of working on HBASE-5313, we want to add a new columnar > encoder/decoder. It makes sense to move compression to be part of > encoder/decoder: > 1) a scanner for a columnar encoded block can do lazy decompression to a > specific part of a key value object > 2) avoid an extra bytes copy from encoder to hblock-writer. > If there is no encoder specified for a writer, the HBlock.Writer will use a > default compression-context to do something very similar to today's code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5155) ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted
[
https://issues.apache.org/jira/browse/HBASE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231637#comment-13231637
]
Hudson commented on HBASE-5155:
---
Integrated in HBase-0.94 #36 (See
[https://builds.apache.org/job/HBase-0.94/36/])
HBASE-5206 port HBASE-5155 to 0.94 (Ashutosh Jindal) (Revision 1301737)
Result = SUCCESS
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKTable.java
> ServerShutDownHandler And Disable/Delete should not happen parallely leading
> to recreation of regions that were deleted
> ---
>
> Key: HBASE-5155
> URL: https://issues.apache.org/jira/browse/HBASE-5155
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.4
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.90.6
>
> Attachments: HBASE-5155_1.patch, HBASE-5155_2.patch,
> HBASE-5155_3.patch, HBASE-5155_latest.patch, hbase-5155_6.patch
>
>
> ServerShutDownHandler and disable/delete table handler races. This is not an
> issue due to TM.
> -> A regionserver goes down. In our cluster the regionserver holds lot of
> regions.
> -> A region R1 has two daughters D1 and D2.
> -> The ServerShutdownHandler gets called and scans the META and gets all the
> user regions
> -> Parallely a table is disabled. (No problem in this step).
> -> Delete table is done.
> -> The tables and its regions are deleted including R1, D1 and D2.. (So META
> is cleaned)
> -> Now ServerShutdownhandler starts to processTheDeadRegion
> {code}
> if (hri.isOffline() && hri.isSplit()) {
> LOG.debug("Offlined and split region " + hri.getRegionNameAsString() +
> "; checking daughter presence");
> fixupDaughters(result, assignmentManager, catalogTracker);
> {code}
> As part of fixUpDaughters as the daughers D1 and D2 is missing for R1
> {code}
> if (isDaughterMissing(catalogTracker, daughter)) {
> LOG.info("Fixup; missing daughter " + daughter.getRegionNameAsString());
> MetaEditor.addDaughter(catalogTracker, daughter, null);
> // TODO: Log WARN if the regiondir does not exist in the fs. If its not
> // there then something wonky about the split -- things will keep going
> // but could be missing references to parent region.
> // And assign it.
> assignmentManager.assign(daughter, true);
> {code}
> we call assign of the daughers.
> Now after this we again start with the below code.
> {code}
> if (processDeadRegion(e.getKey(), e.getValue(),
> this.services.getAssignmentManager(),
> this.server.getCatalogTracker())) {
> this.services.getAssignmentManager().assign(e.getKey(), true);
> {code}
> Now when the SSH scanned the META it had R1, D1 and D2.
> So as part of the above code D1 and D2 which where assigned by fixUpDaughters
> is again assigned by
> {code}
> this.services.getAssignmentManager().assign(e.getKey(), true);
> {code}
> Thus leading to a zookeeper issue due to bad version and killing the master.
> The important part here is the regions that were deleted are recreated which
> i think is more critical.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5206) Port HBASE-5155 to 0.92, 0.94, and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231636#comment-13231636 ] Hudson commented on HBASE-5206: --- Integrated in HBase-0.94 #36 (See [https://builds.apache.org/job/HBase-0.94/36/]) HBASE-5206 port HBASE-5155 to 0.94 (Ashutosh Jindal) (Revision 1301737) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKTable.java > Port HBASE-5155 to 0.92, 0.94, and TRUNK > > > Key: HBASE-5206 > URL: https://issues.apache.org/jira/browse/HBASE-5206 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.2, 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Ashutosh Jindal > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: 5206_92_1.patch, 5206_92_latest_1.patch, > 5206_92_latest_2.patch, 5206_92_latest_3.patch, 5206_trunk_1.patch, > 5206_trunk_latest_1.patch, 5206_trunk_latest_2.patch, > 5206_trunk_latest_3.patch > > > This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should > not happen parallely leading to recreation of regions that were deleted) to > 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5206) Port HBASE-5155 to 0.92, 0.94, and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231619#comment-13231619 ] Zhihong Yu commented on HBASE-5206: --- Integrated to 0.94 as well. > Port HBASE-5155 to 0.92, 0.94, and TRUNK > > > Key: HBASE-5206 > URL: https://issues.apache.org/jira/browse/HBASE-5206 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.2, 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Ashutosh Jindal > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: 5206_92_1.patch, 5206_92_latest_1.patch, > 5206_92_latest_2.patch, 5206_92_latest_3.patch, 5206_trunk_1.patch, > 5206_trunk_latest_1.patch, 5206_trunk_latest_2.patch, > 5206_trunk_latest_3.patch > > > This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should > not happen parallely leading to recreation of regions that were deleted) to > 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5575) Configure Arcanist lint engine for HBase
[ https://issues.apache.org/jira/browse/HBASE-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231613#comment-13231613 ] Phabricator commented on HBASE-5575: mbautin has committed the revision "[jira] [HBASE-5575] Configure Arcanist lint engine for HBase". REVISION DETAIL https://reviews.facebook.net/D2289 COMMIT https://reviews.facebook.net/rHBASE1301751 > Configure Arcanist lint engine for HBase > > > Key: HBASE-5575 > URL: https://issues.apache.org/jira/browse/HBASE-5575 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: Enabling-lint-2012-03-16_13_40_37.patch > > > We need to enable Arcanist lint engine in HBase, so that a commit could be > checked by running "arc lint". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.
[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231610#comment-13231610
]
nkeywal commented on HBASE-5569:
fwiw, I still have the error on testRowMutationMultiThreads, after a few
hundreds iterations... Same logs as above.
> Do not collect deleted KVs when they are still in use by a scanner.
> ---
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5593) Reverse DNS resolution in regionServerStartup() does not strip trailing dot
[ https://issues.apache.org/jira/browse/HBASE-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231609#comment-13231609 ] Zhihong Yu commented on HBASE-5593: --- I meant creating test case for HBaseServer.getRemoteIp().getHostName() returning a trailing dot. > Reverse DNS resolution in regionServerStartup() does not strip trailing dot > --- > > Key: HBASE-5593 > URL: https://issues.apache.org/jira/browse/HBASE-5593 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.6 >Reporter: David S. Wang >Assignee: David S. Wang > Fix For: 0.90.7 > > Attachments: HBASE-5593.patch > > > HBASE-4109 covered the removal of trailing dots in PTR records from reverse > DNS lookups. We seem to have missed a case in HMaster#regionServerStartup(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5575) Configure Arcanist lint engine for HBase
[ https://issues.apache.org/jira/browse/HBASE-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231604#comment-13231604 ] Mikhail Bautin commented on HBASE-5575: --- Reviewed at https://reviews.facebook.net/D2289. > Configure Arcanist lint engine for HBase > > > Key: HBASE-5575 > URL: https://issues.apache.org/jira/browse/HBASE-5575 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: Enabling-lint-2012-03-16_13_40_37.patch > > > We need to enable Arcanist lint engine in HBase, so that a commit could be > checked by running "arc lint". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5575) Configure Arcanist lint engine for HBase
[ https://issues.apache.org/jira/browse/HBASE-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin resolved HBASE-5575. --- Resolution: Fixed Committed to trunk. > Configure Arcanist lint engine for HBase > > > Key: HBASE-5575 > URL: https://issues.apache.org/jira/browse/HBASE-5575 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: Enabling-lint-2012-03-16_13_40_37.patch > > > We need to enable Arcanist lint engine in HBase, so that a commit could be > checked by running "arc lint". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5575) Configure Arcanist lint engine for HBase
[ https://issues.apache.org/jira/browse/HBASE-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-5575: -- Attachment: Enabling-lint-2012-03-16_13_40_37.patch > Configure Arcanist lint engine for HBase > > > Key: HBASE-5575 > URL: https://issues.apache.org/jira/browse/HBASE-5575 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: Enabling-lint-2012-03-16_13_40_37.patch > > > We need to enable Arcanist lint engine in HBase, so that a commit could be > checked by running "arc lint". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231602#comment-13231602
]
Hudson commented on HBASE-5592:
---
Integrated in HBase-0.94 #35 (See
[https://builds.apache.org/job/HBase-0.94/35/])
HBASE-5592 Make it easier to get a table from shell (Revision 1301716)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/ruby/hbase/table.rb
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5581) Creating a table with invalid syntax does not give an error message when it fails
[
https://issues.apache.org/jira/browse/HBASE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231600#comment-13231600
]
Mikhail Bautin commented on HBASE-5581:
---
Binu: thanks for the patch!
Stack: thanks for committing!
> Creating a table with invalid syntax does not give an error message when it
> fails
> -
>
> Key: HBASE-5581
> URL: https://issues.apache.org/jira/browse/HBASE-5581
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Binu John
>Priority: Minor
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5581trunk.patch, D2343.1.patch
>
>
> Creating a table with invalid syntax does not give an error message when it
> fails. In this case, it doesn't actually create the CF requested, but doesn't
> give any indication to the user that it failed.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true, NUMREGIONS
> => 20, SPLITALGO => "HexStringSplit", COMPRESSION => 'LZO', BLOOMFILTER =>
> 'ROW'}
> 0 row(s) in 3.0930 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => []}
> true
> 1 row(s) in 0.1430 seconds
>
> Putting {NUMREGIONS => 20, SPLITALGO => "HexStringSplit"} into a separate
> stanza works fine, so the feature is fine.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true,
> COMPRESSION => 'LZO', BLOOMFILTER => 'ROW'}, {NUMREGIONS => 20, SPLITALGO =>
> "HexStringSplit"}
> 0 row(s) in 2.7860 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => [{NAME => 'test', DATA_BLOCK_ENCODING =>
> 'NONE', true
> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'LZO',
> VERSIONS
> => '1', TTL => '2147483647', BLOCKSIZE => '65536', BLOOMFILTER_ERRORRATE =>
> '
> 0.01', ENCODE_ON_DISK => 'true', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}]}
>
> We should throw an error if we can't create the CF so it's clear to the user.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5549: --- Status: Patch Available (was: Open) > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5549: --- Attachment: 5549.v11.patch > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231598#comment-13231598 ] nkeywal commented on HBASE-5549: v11 with the comments taken into account... Thank you for the review. > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5549: --- Status: Open (was: Patch Available) > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, zookeeper >Affects Versions: 0.96.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, > 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231589#comment-13231589 ] Hadoop QA commented on HBASE-5578: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518726/5589.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 162 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestKeepDeletes org.apache.hadoop.hbase.regionserver.TestMinVersions org.apache.hadoop.hbase.regionserver.TestCompaction Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1209//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1209//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1209//console This message is automatically generated. > NPE when regionserver reported server load, caused rs stop. > --- > > Key: HBASE-5578 > URL: https://issues.apache.org/jira/browse/HBASE-5578 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0 > Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 >Reporter: Storm Lee >Priority: Critical > Fix For: 0.92.2 > > Attachments: 5589.txt > > > The regeionserver log: > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > data3,60020,1331286604591: Unhandled exception: null > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) > at java.lang.Thread.run(Thread.java:662) > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: [] > 2012-03-11 11:55:37,808 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: > requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, > numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, > totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, > memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, > compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, > blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, > blockCacheHitCount=87713, blockCacheMissCount=22144560, > blockCacheEvictedCount=122, blockCacheHitRatio=0%, > blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 > 2012-03-11 11:55:37,992 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled > exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5206) Port HBASE-5155 to 0.92, 0.94, and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231577#comment-13231577 ] Hudson commented on HBASE-5206: --- Integrated in HBase-TRUNK #2685 (See [https://builds.apache.org/job/HBase-TRUNK/2685/]) HBASE-5206 Port HBASE-5155 to trunk (Ashutosh Jindal) (Revision 1301709) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKTable.java > Port HBASE-5155 to 0.92, 0.94, and TRUNK > > > Key: HBASE-5206 > URL: https://issues.apache.org/jira/browse/HBASE-5206 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.2, 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Ashutosh Jindal > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: 5206_92_1.patch, 5206_92_latest_1.patch, > 5206_92_latest_2.patch, 5206_92_latest_3.patch, 5206_trunk_1.patch, > 5206_trunk_latest_1.patch, 5206_trunk_latest_2.patch, > 5206_trunk_latest_3.patch > > > This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should > not happen parallely leading to recreation of regions that were deleted) to > 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5155) ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted
[
https://issues.apache.org/jira/browse/HBASE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231579#comment-13231579
]
Hudson commented on HBASE-5155:
---
Integrated in HBase-TRUNK #2685 (See
[https://builds.apache.org/job/HBase-TRUNK/2685/])
HBASE-5206 Port HBASE-5155 to trunk (Ashutosh Jindal) (Revision 1301709)
Result = SUCCESS
tedyu :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKTable.java
> ServerShutDownHandler And Disable/Delete should not happen parallely leading
> to recreation of regions that were deleted
> ---
>
> Key: HBASE-5155
> URL: https://issues.apache.org/jira/browse/HBASE-5155
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.4
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.90.6
>
> Attachments: HBASE-5155_1.patch, HBASE-5155_2.patch,
> HBASE-5155_3.patch, HBASE-5155_latest.patch, hbase-5155_6.patch
>
>
> ServerShutDownHandler and disable/delete table handler races. This is not an
> issue due to TM.
> -> A regionserver goes down. In our cluster the regionserver holds lot of
> regions.
> -> A region R1 has two daughters D1 and D2.
> -> The ServerShutdownHandler gets called and scans the META and gets all the
> user regions
> -> Parallely a table is disabled. (No problem in this step).
> -> Delete table is done.
> -> The tables and its regions are deleted including R1, D1 and D2.. (So META
> is cleaned)
> -> Now ServerShutdownhandler starts to processTheDeadRegion
> {code}
> if (hri.isOffline() && hri.isSplit()) {
> LOG.debug("Offlined and split region " + hri.getRegionNameAsString() +
> "; checking daughter presence");
> fixupDaughters(result, assignmentManager, catalogTracker);
> {code}
> As part of fixUpDaughters as the daughers D1 and D2 is missing for R1
> {code}
> if (isDaughterMissing(catalogTracker, daughter)) {
> LOG.info("Fixup; missing daughter " + daughter.getRegionNameAsString());
> MetaEditor.addDaughter(catalogTracker, daughter, null);
> // TODO: Log WARN if the regiondir does not exist in the fs. If its not
> // there then something wonky about the split -- things will keep going
> // but could be missing references to parent region.
> // And assign it.
> assignmentManager.assign(daughter, true);
> {code}
> we call assign of the daughers.
> Now after this we again start with the below code.
> {code}
> if (processDeadRegion(e.getKey(), e.getValue(),
> this.services.getAssignmentManager(),
> this.server.getCatalogTracker())) {
> this.services.getAssignmentManager().assign(e.getKey(), true);
> {code}
> Now when the SSH scanned the META it had R1, D1 and D2.
> So as part of the above code D1 and D2 which where assigned by fixUpDaughters
> is again assigned by
> {code}
> this.services.getAssignmentManager().assign(e.getKey(), true);
> {code}
> Thus leading to a zookeeper issue due to bad version and killing the master.
> The important part here is the regions that were deleted are recreated which
> i think is more critical.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231580#comment-13231580
]
Jesse Yates commented on HBASE-5592:
This is a duplicate of 5548 and a worse way to go about it. Prefer to revert
this and go with the impl in 5548 (a little biased here, as I'm working on
5548).
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5581) Creating a table with invalid syntax does not give an error message when it fails
[
https://issues.apache.org/jira/browse/HBASE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231578#comment-13231578
]
Hudson commented on HBASE-5581:
---
Integrated in HBase-TRUNK #2685 (See
[https://builds.apache.org/job/HBase-TRUNK/2685/])
HBASE-5581 Creating a table with invalid syntax does not give an error
message when it fails (Revision 1301688)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/ruby/hbase/admin.rb
> Creating a table with invalid syntax does not give an error message when it
> fails
> -
>
> Key: HBASE-5581
> URL: https://issues.apache.org/jira/browse/HBASE-5581
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Binu John
>Priority: Minor
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5581trunk.patch, D2343.1.patch
>
>
> Creating a table with invalid syntax does not give an error message when it
> fails. In this case, it doesn't actually create the CF requested, but doesn't
> give any indication to the user that it failed.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true, NUMREGIONS
> => 20, SPLITALGO => "HexStringSplit", COMPRESSION => 'LZO', BLOOMFILTER =>
> 'ROW'}
> 0 row(s) in 3.0930 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => []}
> true
> 1 row(s) in 0.1430 seconds
>
> Putting {NUMREGIONS => 20, SPLITALGO => "HexStringSplit"} into a separate
> stanza works fine, so the feature is fine.
> create 'test', {NAME => 'test', VERSIONS => 1, BLOCKCACHE => true,
> COMPRESSION => 'LZO', BLOOMFILTER => 'ROW'}, {NUMREGIONS => 20, SPLITALGO =>
> "HexStringSplit"}
> 0 row(s) in 2.7860 seconds
> hbase(main):002:0> describe 'test'
> DESCRIPTION
> ENABLED
> {NAME => 'test', FAMILIES => [{NAME => 'test', DATA_BLOCK_ENCODING =>
> 'NONE', true
> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'LZO',
> VERSIONS
> => '1', TTL => '2147483647', BLOCKSIZE => '65536', BLOOMFILTER_ERRORRATE =>
> '
> 0.01', ENCODE_ON_DISK => 'true', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}]}
>
> We should throw an error if we can't create the CF so it's clear to the user.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Status: Patch Available (was: Open) > NPE when regionserver reported server load, caused rs stop. > --- > > Key: HBASE-5578 > URL: https://issues.apache.org/jira/browse/HBASE-5578 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0 > Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 >Reporter: Storm Lee >Priority: Critical > Attachments: 5589.txt > > > The regeionserver log: > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > data3,60020,1331286604591: Unhandled exception: null > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) > at java.lang.Thread.run(Thread.java:662) > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: [] > 2012-03-11 11:55:37,808 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: > requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, > numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, > totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, > memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, > compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, > blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, > blockCacheHitCount=87713, blockCacheMissCount=22144560, > blockCacheEvictedCount=122, blockCacheHitRatio=0%, > blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 > 2012-03-11 11:55:37,992 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled > exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Fix Version/s: 0.92.2 > NPE when regionserver reported server load, caused rs stop. > --- > > Key: HBASE-5578 > URL: https://issues.apache.org/jira/browse/HBASE-5578 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0 > Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 >Reporter: Storm Lee >Priority: Critical > Fix For: 0.92.2 > > Attachments: 5589.txt > > > The regeionserver log: > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > data3,60020,1331286604591: Unhandled exception: null > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) > at java.lang.Thread.run(Thread.java:662) > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: [] > 2012-03-11 11:55:37,808 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: > requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, > numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, > totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, > memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, > compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, > blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, > blockCacheHitCount=87713, blockCacheMissCount=22144560, > blockCacheEvictedCount=122, blockCacheHitRatio=0%, > blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 > 2012-03-11 11:55:37,992 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled > exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Attachment: 5589.txt How about this. Goes through Store and checks all Reader instances for null before using. We were doing this in half the cases already. Converts the NPE into a null warning. Means we don't crash. Puts off having to spend time on why the Reader is null at particular junctures. Should go into 0.94? > NPE when regionserver reported server load, caused rs stop. > --- > > Key: HBASE-5578 > URL: https://issues.apache.org/jira/browse/HBASE-5578 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0 > Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 >Reporter: Storm Lee >Priority: Critical > Fix For: 0.92.2 > > Attachments: 5589.txt > > > The regeionserver log: > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > data3,60020,1331286604591: Unhandled exception: null > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) > at java.lang.Thread.run(Thread.java:662) > 2012-03-11 11:55:37,808 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: [] > 2012-03-11 11:55:37,808 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: > requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, > numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, > totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, > memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, > compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, > blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, > blockCacheHitCount=87713, blockCacheMissCount=22144560, > blockCacheEvictedCount=122, blockCacheHitRatio=0%, > blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 > 2012-03-11 11:55:37,992 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled > exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231557#comment-13231557 ] stack commented on HBASE-5563: -- +1 on patch for trunk and 0.92 (again). Older regionids should appear earlier in a sorted list than newer regionids as per this patch. > HRegionInfo#compareTo add the comparison of regionId > > > Key: HBASE-5563 > URL: https://issues.apache.org/jira/browse/HBASE-5563 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-5563.patch, HBASE-5563v2.patch, > HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch > > > In the one region multi assigned case, we could find that two regions have > the same table name, same startKey, same endKey, and different regionId, so > these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5592) Make it easier to get a table from shell
[
https://issues.apache.org/jira/browse/HBASE-5592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-5592.
--
Resolution: Fixed
Fix Version/s: 0.92.2
Hadoop Flags: Reviewed
Committed trunk, 0.94, and 0.92 branches. Thanks for the patch Ben.
> Make it easier to get a table from shell
>
>
> Key: HBASE-5592
> URL: https://issues.apache.org/jira/browse/HBASE-5592
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Ben West
>Assignee: Ben West
>Priority: Trivial
> Labels: shell
> Fix For: 0.92.2, 0.94.0
>
> Attachments: publicTable.patch
>
>
> The one argument constructor to HTable was removed at some point, which means
> that you now have to pass in a Configuration to instantiate an HTable. This
> is annoying for me when I create quick scripts.
> This JIRA is a tiny patch which lets you get an HTable instance in the shell
> by doing
> {code}foo_table = @shell.hbase_table('foo').table{code}
> Basically, it is changing table to be a public member rather than a private
> one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5588) Deprecate/remove AssignmentManager#clearRegionFromTransition
[ https://issues.apache.org/jira/browse/HBASE-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231553#comment-13231553 ] stack commented on HBASE-5588: -- +1 on patch set > Deprecate/remove AssignmentManager#clearRegionFromTransition > > > Key: HBASE-5588 > URL: https://issues.apache.org/jira/browse/HBASE-5588 > Project: HBase > Issue Type: Sub-task > Components: hbck >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: hbase-5588-0.90.patch, hbase-5588-0.94.patch, > hbase-5588.patch > > > This method is essentially a dupe of Assignment#regionOffline. As suggested > in early review of HBASE-5128 - deprecate up to 0.94 and remove from > 0.96/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5593) Reverse DNS resolution in regionServerStartup() does not strip trailing dot
[ https://issues.apache.org/jira/browse/HBASE-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231551#comment-13231551 ] stack commented on HBASE-5593: -- @Ted You mean a unit test for the Strings.domainNamePointerToHostName facility? (We don't want to test InetSocketAddress). > Reverse DNS resolution in regionServerStartup() does not strip trailing dot > --- > > Key: HBASE-5593 > URL: https://issues.apache.org/jira/browse/HBASE-5593 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.6 >Reporter: David S. Wang >Assignee: David S. Wang > Fix For: 0.90.7 > > Attachments: HBASE-5593.patch > > > HBASE-4109 covered the removal of trailing dots in PTR records from reverse > DNS lookups. We seem to have missed a case in HMaster#regionServerStartup(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5568) Multi concurrent flushcache() for one region could cause data loss
[ https://issues.apache.org/jira/browse/HBASE-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231540#comment-13231540 ] Hudson commented on HBASE-5568: --- Integrated in HBase-0.94 #34 (See [https://builds.apache.org/job/HBase-0.94/34/]) HBASE-5568 Multi concurrent flushcache() for one region could cause data loss (Chunhui) (Revision 1301676) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java > Multi concurrent flushcache() for one region could cause data loss > -- > > Key: HBASE-5568 > URL: https://issues.apache.org/jira/browse/HBASE-5568 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5568-90.patch, HBASE-5568.patch, HBASE-5568.patch, > HBASE-5568v2.patch > > > We could call HRegion#flushcache() concurrently now through > HRegionServer#splitRegion or HRegionServer#flushRegion by HBaseAdmin. > However, we find if HRegion#internalFlushcache() is called concurrently by > multi thread, HRegion.memstoreSize will be calculated wrong. > At the end of HRegion#internalFlushcache(), we will do > this.addAndGetGlobalMemstoreSize(-flushsize), but the flushsize may not the > actual memsize which flushed to hdfs. It cause HRegion.memstoreSize is > negative and prevent next flush if we close this region. > Logs in RS for region e9d827913a056e696c39bc569ea3 > 2012-03-11 16:31:36,690 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 128.0m > 2012-03-11 16:31:37,999 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/8162481165586107427, entries=153106, sequenceid=619316544, > memsize=59.6m, filesize=31.2m > 2012-03-11 16:31:38,830 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 134.8m > 2012-03-11 16:31:39,458 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/3425971951499794221, entries=230183, sequenceid=619316544, > memsize=68.5m, filesize=26.6m > 2012-03-11 16:31:39,459 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~128.1m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 2769ms, sequenceid=619316544, compaction > requested=false > 2012-03-11 16:31:39,459 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > Started memstore flush for > writetest1,,1331454657410.e9d827913a056e696c39bc569ea3 > f99f., current region memstore size 6.8m > 2012-03-11 16:31:39,529 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/1811012969998104626, entries=8002, sequenceid=619332759, > memsize=3.1m, filesize=1.6m > 2012-03-11 16:31:39,640 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/770333473623552048, entries=12231, sequenceid=619332759, > memsize=3.6m, filesize=1.4m > 2012-03-11 16:31:39,641 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~134.8m for region > writetest1,,1331454657410.e9d827913a > 056e696c39bc569ea3f99f. in 811ms, sequenceid=619332759, compaction > requested=true > 2012-03-11 16:31:39,707 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf1/5656568849587368557, entries=119, sequenceid=619332979, > memsize=47.4k, filesize=25.6k > 2012-03-11 16:31:39,775 INFO org.apache.hadoop.hbase.regionserver.Store: > Added > hdfs://dw74.kgb.sqa.cm4:9700/hbase-func1/writetest1/e9d827913a056e696c39bc569e > a3f99f/cf2/794343845650987521, entries=157, sequenceid=619332979, > memsize=47.8k, filesize=19.3k > 2012-03-11 16:31:39,777 INFO org.apache.hadoop.hbase.regionserver.HRegion: > Finished memstore flush of ~6.8m for region > writetest1,,1331454657410.e9d827913a05 > 6e696c39bc569ea3f99f. in 318ms, sequenceid=619332979, compaction > requested=true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.
