[jira] [Commented] (HBASE-4605) Constraints

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159153#comment-13159153
 ] 

jirapos...@reviews.apache.org commented on HBASE-4605:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3551
---


Most of what I see is just style issues, and a number of the HTableDescriptor 
changes seem unnecessary.  If we clean up those, and assuming the latest patch 
has some documentation, I think this is looking pretty good.


src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
https://reviews.apache.org/r/2579/#comment7921

Nice addition to have



src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
https://reviews.apache.org/r/2579/#comment7913

I don't see why these changes are necessary when I don't see them being 
used anywhere outside of HTD?  Do these additions buy us anything?  In my 
opinion, they're not making the code any clearer than the current 
Byte.toString(), etc.



src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
https://reviews.apache.org/r/2579/#comment7916

style nit: if body should be contained in braces



src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
https://reviews.apache.org/r/2579/#comment7914

Add constraints.size() to this message and you don't need the separate 
LOG.debug() message.



src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
https://reviews.apache.org/r/2579/#comment7912

style nit: for loop body should be enclosed in braces



src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
https://reviews.apache.org/r/2579/#comment7917

I see the usage outside of HTD now, but still don't see where this provides 
more utility than Bytes.toString()



src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
https://reviews.apache.org/r/2579/#comment7918

Be consistent with braces here



src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
https://reviews.apache.org/r/2579/#comment7915

if / else bodies should be contained in braces.



src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
https://reviews.apache.org/r/2579/#comment7922

This still seems odd to me.  I can't see storing an integer value encoded 
as a string, in order to be able to check for it being a valid integer.

I would think most users would be storing integer values using 
Bytes.toInt(), for which the checks here would not work.



src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
https://reviews.apache.org/r/2579/#comment7919

This differs from what seems to be the latest patch in JIRA 
(java_HBASE-4605_v3.txt).  Are there other differences than the javadoc?

For a patch of this size, review still belongs here in review board (even 
with HadoopQA patch testing), so we should keep both versions in sync.  Please 
update the diff here with the latest patch.



src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java
https://reviews.apache.org/r/2579/#comment7920

brace alignment and indentation


- Gary


On 2011-11-23 21:19:56, Jesse Yates wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2579/
bq.  ---
bq.  
bq.  (Updated 2011-11-23 21:19:56)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Most of the implementation for adding constraints as a coprocessor. 
bq.  
bq.  Looking for general comments on style/structure, though nitpicks are ok 
too. 
bq.  
bq.  Currently missing implementation for disableConstraints() since that will 
require adding removeCoprocessor() to HTD (also comments on if this is worth it 
would be good). 
bq.  
bq.  
bq.  This addresses bug HBASE-4605.
bq.  https://issues.apache.org/jira/browse/HBASE-4605
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a 
bq.src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java 
PRE-CREATION 
bq.

[jira] [Commented] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159164#comment-13159164
 ] 

Hudson commented on HBASE-4885:
---

Integrated in HBase-TRUNK #2496 (See 
[https://builds.apache.org/job/HBase-TRUNK/2496/])
HBASE-4885 Building against Hadoop 0.23 uses out-of-date MapReduce artifacts

stack : 
Files : 
* /hbase/trunk/pom.xml
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java


 Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
 -

 Key: HBASE-4885
 URL: https://issues.apache.org/jira/browse/HBASE-4885
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Tom White
Assignee: Tom White
 Fix For: 0.94.0

 Attachments: HBASE-4885.patch


 The hadoop-mapred artifacts have been replaced by hadoop-mapreduce-* 
 artifacts in 0.23 onwards.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4884) Allow environment overrides for various HBase processes

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159163#comment-13159163
 ] 

Hudson commented on HBASE-4884:
---

Integrated in HBase-TRUNK #2496 (See 
[https://builds.apache.org/job/HBase-TRUNK/2496/])
HBASE-4884 Allow environment overrides for various HBase processes

stack : 
Files : 
* /hbase/trunk/bin/hbase


 Allow environment overrides for various HBase processes
 ---

 Key: HBASE-4884
 URL: https://issues.apache.org/jira/browse/HBASE-4884
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Ryan Thiessen
Priority: Minor
 Fix For: 0.94.0

 Attachments: hbase-4884.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 The current shell scripts have no mechanism for granting different 
 environments to various HBase subcomponents.  Sometimes we want to customize 
 these, for example to run the thrift command with a lower HBASE_HEAPSIZE than 
 what we grant the regionservers.
 Checking for the presence of an override file and then sourcing it if present 
 allows me to override this heapsize or any other variable.
 Test process:
 * tested with file missing. no problem, used default conf/hbase-env.sh 
 settings.
 * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww 
 showed -Xmx1234m
 * started regionserver using bin/hadoop-daemon.sh
 * started hbase shell
 * ran some jruby scripts via bin/hbase org.jruby.Main
 * running in production for approx 2 weeks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master

2011-11-29 Thread Ted Yu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159180#comment-13159180
 ] 

Ted Yu commented on HBASE-4729:
---

For the unexpected zk exception, we should abort the master instead of logging 
error. 

 Race between online altering and splitting kills the master
 ---

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (HBASE-4729) Race between online altering and splitting kills the master

2011-11-29 Thread Ted Yu (Issue Comment Edited) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159180#comment-13159180
 ] 

Ted Yu edited comment on HBASE-4729 at 11/29/11 3:07 PM:
-

{code}
+} catch (KeeperException ke) {
+  LOG.error(Unexpected zk state, ke);
+}
{code}
For KeeperException.SessionExpiredException, we may optionally follow example 
of tryRecoveringExpiredZKSession() in HMaster.

For unexpected zk exception, we should abort the master instead of logging 
error.

TestAssignmentManager is missing test category.

  was (Author: yuzhih...@gmail.com):
For the unexpected zk exception, we should abort the master instead of 
logging error. 
  
 Race between online altering and splitting kills the master
 ---

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master

2011-11-29 Thread stack (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159311#comment-13159311
 ] 

stack commented on HBASE-4729:
--

bq. For unexpected zk exception, we should abort the master instead of logging 
error.

It does.  It falls through to the abort call.

I'll make issue for SessionExpiredException.

I'll fix missing test category on commit.

But thinking more on this patch, it works for the altering case but I don't 
think it will work for the issue I reported where we are unassigning because 
region is being moved by balancer.   The unassign looks like it moved so we'll 
go ahead and try open region in new location.  The splitting code may fix it up 
but need to check.

 Race between online altering and splitting kills the master
 ---

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4889) HRegionInfo.isMetaTable() should be true for -ROOT- regions

2011-11-29 Thread Created 
HRegionInfo.isMetaTable() should be true for -ROOT- regions
---

 Key: HBASE-4889
 URL: https://issues.apache.org/jira/browse/HBASE-4889
 Project: HBase
  Issue Type: Bug
Reporter: Daniel Gómez Ferro


According to its javadoc, HRegionInfo.isMetaTable() should return true if the 
region is either a .META. or -ROOT- region, but only does so for .META. regions.

The change in behavior happened in HBASE-451

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4889) HRegionInfo.isMetaTable() should be true for -ROOT- regions

2011-11-29 Thread Updated 

 [ 
https://issues.apache.org/jira/browse/HBASE-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Gómez Ferro updated HBASE-4889:
--

Attachment: HBASE-4889.patch

Attach a test case and a possible fix.

The patch also uses .isMetaTable() when possible.


 HRegionInfo.isMetaTable() should be true for -ROOT- regions
 ---

 Key: HBASE-4889
 URL: https://issues.apache.org/jira/browse/HBASE-4889
 Project: HBase
  Issue Type: Bug
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-4889.patch


 According to its javadoc, HRegionInfo.isMetaTable() should return true if the 
 region is either a .META. or -ROOT- region, but only does so for .META. 
 regions.
 The change in behavior happened in HBASE-451

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4889) HRegionInfo.isMetaTable() should be true for -ROOT- regions

2011-11-29 Thread Updated 

 [ 
https://issues.apache.org/jira/browse/HBASE-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Gómez Ferro updated HBASE-4889:
--

Status: Patch Available  (was: Open)

 HRegionInfo.isMetaTable() should be true for -ROOT- regions
 ---

 Key: HBASE-4889
 URL: https://issues.apache.org/jira/browse/HBASE-4889
 Project: HBase
  Issue Type: Bug
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-4889.patch


 According to its javadoc, HRegionInfo.isMetaTable() should return true if the 
 region is either a .META. or -ROOT- region, but only does so for .META. 
 regions.
 The change in behavior happened in HBASE-451

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4120) isolation and allocation

2011-11-29 Thread Ted Yu (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-4120:
--

Comment: was deleted

(was: -1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505326/TablePrioriy_v9.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 9 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -140 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 84 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.client.TestAdmin

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/393//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/393//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/393//console

This message is automatically generated.)

 isolation and allocation
 

 Key: HBASE-4120
 URL: https://issues.apache.org/jira/browse/HBASE-4120
 Project: HBase
  Issue Type: New Feature
  Components: master, regionserver
Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0
Reporter: Liu Jia
Assignee: Liu Jia
 Fix For: 0.94.0

 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, 
 Design_document_for_HBase_isolation_and_allocation_Revised.pdf, 
 HBase_isolation_and_allocation_user_guide.pdf, 
 Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, 
 TablePriority_v8.patch, TablePriority_v8.patch, 
 TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch


 The HBase isolation and allocation tool is designed to help users manage 
 cluster resource among different application and tables.
 When we have a large scale of HBase cluster with many applications running on 
 it, there will be lots of problems. In Taobao there is a cluster for many 
 departments to test their applications performance, these applications are 
 based on HBase. With one cluster which has 12 servers, there will be only one 
 application running exclusively on this server, and many other applications 
 must wait until the previous test finished.
 After we add allocation manage function to the cluster, applications can 
 share the cluster and run concurrently. Also if the Test Engineer wants to 
 make sure there is no interference, he/she can move out other tables from 
 this group.
 In groups we use table priority to allocate resource, when system is busy; we 
 can make sure high-priority tables are not affected lower-priority tables
 Different groups can have different region server configurations, some groups 
 optimized for reading can have large block cache size, and others optimized 
 for writing can have large memstore size. 
 Tables and region servers can be moved easily between groups; after changing 
 the configuration, a group can be restarted alone instead of restarting the 
 whole cluster.
 git entry : https://github.com/ICT-Ope/HBase_allocation .
 We hope our work is helpful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4890) fix possible NPE in HConnectionManager

2011-11-29 Thread Jonathan Hsieh (Created) (JIRA) 
fix possible NPE in HConnectionManager
--

 Key: HBASE-4890
 URL: https://issues.apache.org/jira/browse/HBASE-4890
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jonathan Hsieh


I was running YCSB against a 0.92 branch and encountered this error message:

{code}
11/11/29 08:47:16 WARN client.HConnectionManager$HConnectionImplementation: 
Failed all from 
region=usertable,user3917479014967760871,1322555655231.f78d161e5724495a9723bcd972f97f41.,
 hostname=c0316.hal.cloudera.com, port=57020
java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1501)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1353)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:898)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:775)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:750)
at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
at com.yahoo.ycsb.DBWrapper.update(Unknown Source)
at com.yahoo.ycsb.workloads.CoreWorkload.doTransactionUpdate(Unknown 
Source)
at com.yahoo.ycsb.workloads.CoreWorkload.doTransaction(Unknown Source)
at com.yahoo.ycsb.ClientThread.run(Unknown Source)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1315)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1327)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1325)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:158)
at $Proxy4.multi(Unknown Source)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1330)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1328)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1309)
... 7 more
{code}

It looks like the NPE is caused by server being null in the MultiRespone call() 
method.

{code}
 public MultiResponse call() throws IOException {
 return getRegionServerWithoutRetries(
 new ServerCallableMultiResponse(connection, tableName, null) {
   public MultiResponse call() throws IOException {
 return server.multi(multi);
   }
   @Override
   public void connect(boolean reload) throws IOException {
 server =
   connection.getHRegionConnection(loc.getHostname(), 
loc.getPort());
   }
 }
 );
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159371#comment-13159371
 ] 

Hudson commented on HBASE-4885:
---

Integrated in HBase-TRUNK-security #14 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/14/])
HBASE-4885 Building against Hadoop 0.23 uses out-of-date MapReduce artifacts

stack : 
Files : 
* /hbase/trunk/pom.xml
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java


 Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
 -

 Key: HBASE-4885
 URL: https://issues.apache.org/jira/browse/HBASE-4885
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Tom White
Assignee: Tom White
 Fix For: 0.94.0

 Attachments: HBASE-4885.patch


 The hadoop-mapred artifacts have been replaced by hadoop-mapreduce-* 
 artifacts in 0.23 onwards.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4884) Allow environment overrides for various HBase processes

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159370#comment-13159370
 ] 

Hudson commented on HBASE-4884:
---

Integrated in HBase-TRUNK-security #14 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/14/])
HBASE-4884 Allow environment overrides for various HBase processes

stack : 
Files : 
* /hbase/trunk/bin/hbase


 Allow environment overrides for various HBase processes
 ---

 Key: HBASE-4884
 URL: https://issues.apache.org/jira/browse/HBASE-4884
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Ryan Thiessen
Priority: Minor
 Fix For: 0.94.0

 Attachments: hbase-4884.txt

   Original Estimate: 0h
  Remaining Estimate: 0h

 The current shell scripts have no mechanism for granting different 
 environments to various HBase subcomponents.  Sometimes we want to customize 
 these, for example to run the thrift command with a lower HBASE_HEAPSIZE than 
 what we grant the regionservers.
 Checking for the presence of an override file and then sourcing it if present 
 allows me to override this heapsize or any other variable.
 Test process:
 * tested with file missing. no problem, used default conf/hbase-env.sh 
 settings.
 * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww 
 showed -Xmx1234m
 * started regionserver using bin/hadoop-daemon.sh
 * started hbase shell
 * ran some jruby scripts via bin/hbase org.jruby.Main
 * running in production for approx 2 weeks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)

2011-11-29 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159391#comment-13159391
 ] 

Phabricator commented on HBASE-4218:


tedyu has commented on the revision [jira] [HBASE-4218] Delta encoding for 
keys in HFile.

  Nice work, Mikhail and Jacek.

  Please add category to the new tests.

  Are there performance numbers for various encoders other than Prefix encoder ?

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java:337 As Matt 
pointed out, the return value should be stored in hfileBlock so that we don't 
incur double encoding.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java:305 Similar 
to the case in HFileReaderV1, return value should be stored in dataBlock.
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoderAlgorithms.java:33
 Matt suggested alternative names for DeltaEncoding:
  KeyValueEncoding, DataBlockEncoding, HCellEncoding, BlockEncoding.

  DataBlockEncoding sounds good.
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DiffKeyDeltaEncoder.java:405
 Misspelling: comperator should be comparator.
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoderAlgorithms.java:65
 Javadoc doesn't match actual class name.
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/FastDiffDeltaEncoder.java:53
 The tail should read '128 bit encoding'
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/FastDiffDeltaEncoder.java:28
 This class is only used locally. It should be an inner class.
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DiffKeyDeltaEncoder.java:49
 Tail should read '128 bit encoding'
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DiffKeyDeltaEncoder.java:346
 Please remove extra blank line.
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DiffKeyDeltaEncoder.java:28
 Please change this class to inner class.
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoderBufferTooSmallException.java:22
 Should read 'which indicates'

REVISION DETAIL
  https://reviews.facebook.net/D447


 Delta Encoding of KeyValues  (aka prefix compression)
 -

 Key: HBASE-4218
 URL: https://issues.apache.org/jira/browse/HBASE-4218
 Project: HBase
  Issue Type: Improvement
  Components: io
Affects Versions: 0.94.0
Reporter: Jacek Migdal
Assignee: Mikhail Bautin
  Labels: compression
 Attachments: D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, 
 D447.5.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff


 A compression for keys. Keys are sorted in HFile and they are usually very 
 similar. Because of that, it is possible to design better compression than 
 general purpose algorithms,
 It is an additional step designed to be used in memory. It aims to save 
 memory in cache as well as speeding seeks within HFileBlocks. It should 
 improve performance a lot, if key lengths are larger than value lengths. For 
 example, it makes a lot of sense to use it when value is a counter.
 Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) 
 shows that I could achieve decent level of compression:
  key compression ratio: 92%
  total compression ratio: 85%
  LZO on the same data: 85%
  LZO after delta encoding: 91%
 While having much better performance (20-80% faster decompression ratio than 
 LZO). Moreover, it should allow far more efficient seeking which should 
 improve performance a bit.
 It seems that a simple compression algorithms are good enough. Most of the 
 savings are due to prefix compression, int128 encoding, timestamp diffs and 
 bitfields to avoid duplication. That way, comparisons of compressed data can 
 be much faster than a byte comparator (thanks to prefix compression and 
 bitfields).
 In order to implement it in HBase two important changes in design will be 
 needed:
 -solidify interface to HFileBlock / HFileReader Scanner to provide seeking 
 and iterating; access to uncompressed buffer in HFileBlock will have bad 
 performance
 -extend comparators to support comparison assuming that N first bytes are 
 equal (or some fields are equal)
 Link to a discussion about something similar:
 http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windowssubj=Re+prefix+compression

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4729) Race between online altering and splitting kills the master

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4729:
-

Attachment: 4729-v5.txt

Added missing annotation.  Otherwise, its pretty much v4.

 Race between online altering and splitting kills the master
 ---

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4867) A tool to merge configuration files

2011-11-29 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159409#comment-13159409
 ] 

Phabricator commented on HBASE-4867:


Karthik has accepted the revision [jira] [HBASE-4867] A tool to merge 
configuration files.

  Looks good!

  Are you planning on using this only for the opensource template on v2 install 
scripts or all of them?

INLINE COMMENTS
  src/main/python/hbase/merge_conf.py:120 Nice!

REVISION DETAIL
  https://reviews.facebook.net/D537


 A tool to merge configuration files
 ---

 Key: HBASE-4867
 URL: https://issues.apache.org/jira/browse/HBASE-4867
 Project: HBase
  Issue Type: New Feature
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D537.1.patch


 With our cluster configuration setup it would be good to have a tool that 
 would merge HBase configuration, so that files appearing later in the list 
 would override properties specified in earlier files. This way we could merge 
 application-specific configuration file with the cluster-specific 
 configuration file (with the latter overriding the former) and produce a 
 single HBase configuration file to install on the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master

2011-11-29 Thread Ted Yu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159410#comment-13159410
 ] 

Ted Yu commented on HBASE-4729:
---

Since TestAssignmentManager would be run in a shared JVM under TRUNK in the 
near future, I think it should be a medium test.

From N:
- Small tests are executed in a shared JVM. We put in this category all the
tests that can be executed quickly (the maximum execution time for a test
is 15 seconds, and they do not use a cluster) in a shared jvm.


 Race between online altering and splitting kills the master
 ---

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4729) Clash between balancer and splitting kills the master

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4729:
-

Summary: Clash between balancer and splitting kills the master  (was: Race 
between online altering and splitting kills the master)

Updated the subject

 Clash between balancer and splitting kills the master
 -

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master

2011-11-29 Thread stack (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159416#comment-13159416
 ] 

stack commented on HBASE-4729:
--

Now I think we should commit this.

My concern about a failed unassign as part of a balance is unfounded.  Balances 
work by running unassign and then in the ClosedRegionHandler we'll do the 
assign elsewhere.  The latter assign will not happen if the unassign/close 
doesn't complete because of a concurrent SPLITTING/SPLIT.

Regards the TODO on what to do on rollback of a failed split, again, we should 
be ok in the balancing case; the cluster will be out-of-balance if parent comes 
back on line after our failed unassign attempt but so what.  It'll be fixed 
next time the balancer runs. 

On SessionExpiredException handling, that should be done in one place only 
rather than spread about the codebase (We need to refactor RecoverableZooKeeper 
so that it will create new ZooKeeper to retry on SessionExpiredException).

Chatting with J-D, I should rename this issue since the focus has become 
balance+concurrent split; the alter case is less critical since we do not have 
online alter enabled by default (this patch will help some but more work to do 
-- I'll open new issue).

 Race between online altering and splitting kills the master
 ---

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on

[jira] [Updated] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4729:
-

Summary: Clash between region unassign and splitting kills the master  
(was: Clash between balancer and splitting kills the master)

J-D didn't like my last attempt at at title.  Making him happy.

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread Jean-Daniel Cryans (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159425#comment-13159425
 ] 

Jean-Daniel Cryans commented on HBASE-4729:
---

+1 on v5, some comments:

 - I would have preferred to see the debugLog changes in another patch, makes 
it harder to review (but I agree with that change). Same for the other added 
javadocs and whatnot.
 - Minor nit, there's a white space missing before the path:

bq. LOG.warn(Failed getData on SPLITTING/SPLIT at + path +

 - In the same log message, I'm not sure what a user would be supposed to do 
with the following:

bq. encodedName + , no longer exists -- confirm, ke);

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread Ted Yu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159426#comment-13159426
 ] 

Ted Yu commented on HBASE-4729:
---

+1 on patch v5.

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4889) HRegionInfo.isMetaTable() should be true for -ROOT- regions

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4889:
-

   Resolution: Fixed
Fix Version/s: 0.92.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Thanks for fixing regression Daniel.  Committed to trunk and 0.92 branch.

 HRegionInfo.isMetaTable() should be true for -ROOT- regions
 ---

 Key: HBASE-4889
 URL: https://issues.apache.org/jira/browse/HBASE-4889
 Project: HBase
  Issue Type: Bug
Reporter: Daniel Gómez Ferro
 Fix For: 0.92.0

 Attachments: HBASE-4889.patch


 According to its javadoc, HRegionInfo.isMetaTable() should return true if the 
 region is either a .META. or -ROOT- region, but only does so for .META. 
 regions.
 The change in behavior happened in HBASE-451

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4867) A tool to merge configuration files

2011-11-29 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159433#comment-13159433
 ] 

Phabricator commented on HBASE-4867:


mbautin has commented on the revision [jira] [HBASE-4867] A tool to merge 
configuration files.

  This is only necessary for the open-source HBase trunk (0.89-fb can read 
hbase-site-custom.xml). However, nothing prevents us from using this approach 
for all of our dev clusters.

REVISION DETAIL
  https://reviews.facebook.net/D537


 A tool to merge configuration files
 ---

 Key: HBASE-4867
 URL: https://issues.apache.org/jira/browse/HBASE-4867
 Project: HBase
  Issue Type: New Feature
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D537.1.patch


 With our cluster configuration setup it would be good to have a tool that 
 would merge HBase configuration, so that files appearing later in the list 
 would override properties specified in earlier files. This way we could merge 
 application-specific configuration file with the cluster-specific 
 configuration file (with the latter overriding the former) and produce a 
 single HBase configuration file to install on the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread stack (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159432#comment-13159432
 ] 

stack commented on HBASE-4729:
--

@J-D Will fix on commit.  On log message, its intentionally irritating.  I want 
to leave it so until the presumption made here is verified (This should be very 
rare -- if not, there is a problem we  need to revisit).

@Ted Thanks for reviews.

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4891) HTable.ClientScanner needs to clone the Scan object

2011-11-29 Thread Jean-Daniel Cryans (Created) (JIRA) 
HTable.ClientScanner needs to clone the Scan object
---

 Key: HBASE-4891
 URL: https://issues.apache.org/jira/browse/HBASE-4891
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
 Fix For: 0.92.0, 0.94.0, 0.90.5


The ClientScanner can make some changes to the Scan object like setting a new 
start row when it gets an error, so if someone was reusing that object for a 
new scan thinking it would have the same properties, he/she would currently be 
wrong.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4729:
-

Attachment: 4729-v6-trunk.txt

What I applied trunk.

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729-v6-trunk.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread Hadoop QA (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159443#comment-13159443
 ] 

Hadoop QA commented on HBASE-4729:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12505515/4729-v6-trunk.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -162 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 68 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/405//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/405//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/405//console

This message is automatically generated.

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729-v6-trunk.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:

[jira] [Updated] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4729:
-

Attachment: 4729-v6-092.txt

What I applied to 092 branch (There is no categorization of tests in 0.92).

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729-v6-092.txt, 4729-v6-trunk.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4729:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed trunk and branch (thanks for reviews lads).

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729-v6-092.txt, 4729-v6-trunk.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4892) [book] book updates 11-29-2011

2011-11-29 Thread Doug Meil (Created) (JIRA) 
[book] book updates 11-29-2011
--

 Key: HBASE-4892
 URL: https://issues.apache.org/jira/browse/HBASE-4892
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor


book.xml
* MR chapter, added using HBase table as reducer  
* data model, added info about different types of tombstones per conversation 
this week about deletes.

ops_mgt.xml
* added region mgt section, mentioned major compation and region merges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4888) Not close ResultScanner cause the cluster abnormal ( RS memory increase)

2011-11-29 Thread stack (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159459#comment-13159459
 ] 

stack commented on HBASE-4888:
--

Do you have evidence this is so Yuan?  Thank you.

 Not close ResultScanner cause the cluster abnormal ( RS memory increase)
 

 Key: HBASE-4888
 URL: https://issues.apache.org/jira/browse/HBASE-4888
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.3
 Environment: CentOS 5.5 final, 
 hadoop-0.20.2-cdh3u1,hbase-0.20.2-cdh3u1
Reporter: Yuan Kang
  Labels: ResultScanner
   Original Estimate: 672h
  Remaining Estimate: 672h

 A ResultScanner is created in client side, If the user doesn't invoke the 
 ResultScanner.close() ,it happens that the memory of RegionServer increase 
 rapidly and hold for a long time. Finally,the cluster goes to an abnormal 
 status
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4892) [book] book updates 11-29-2011

2011-11-29 Thread Doug Meil (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4892:
-

Attachment: docbkx_HBASE_4892.patch

 [book] book updates 11-29-2011
 --

 Key: HBASE-4892
 URL: https://issues.apache.org/jira/browse/HBASE-4892
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4892.patch


 book.xml
 * MR chapter, added using HBase table as reducer  
 * data model, added info about different types of tombstones per conversation 
 this week about deletes.
 ops_mgt.xml
 * added region mgt section, mentioned major compation and region merges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4892) [book] book updates 11-29-2011

2011-11-29 Thread Doug Meil (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4892:
-

Status: Patch Available  (was: Open)

 [book] book updates 11-29-2011
 --

 Key: HBASE-4892
 URL: https://issues.apache.org/jira/browse/HBASE-4892
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4892.patch


 book.xml
 * MR chapter, added using HBase table as reducer  
 * data model, added info about different types of tombstones per conversation 
 this week about deletes.
 ops_mgt.xml
 * added region mgt section, mentioned major compation and region merges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4892) [book] book updates 11-29-2011

2011-11-29 Thread Doug Meil (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4892:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] book updates 11-29-2011
 --

 Key: HBASE-4892
 URL: https://issues.apache.org/jira/browse/HBASE-4892
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4892.patch


 book.xml
 * MR chapter, added using HBase table as reducer  
 * data model, added info about different types of tombstones per conversation 
 this week about deletes.
 ops_mgt.xml
 * added region mgt section, mentioned major compation and region merges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4893) HConnectionImplementation closed-but-not-deleted, need a way to find the state of connection

2011-11-29 Thread Mubarak Seyed (Created) (JIRA) 
HConnectionImplementation closed-but-not-deleted, need a way to find the state 
of connection


 Key: HBASE-4893
 URL: https://issues.apache.org/jira/browse/HBASE-4893
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.4, 0.90.3, 0.90.2, 0.90.1
 Environment: Linux 2.6, HBase-0.90.1
Reporter: Mubarak Seyed
 Fix For: 0.90.5


In abort() of HConnectionManager$HConnectionImplementation, instance of 
HConnectionImplementation is marked as this.closed=true.

There is no way for client application to check the hbase client connection 
whether it is still opened/good (this.closed=false) or not. We need a method to 
validate the state of a connection like isClosed().

{code}
public boolean isClosed(){
   return this.closed;
} 
{code}

Once the connection is closed and it is not deleted. Client application still 
gets a connection from HConnectionManager.getConnection(Configuration) and 
tries to make a RPC call to RS, since connection is already closed, 
HConnectionImplementation.getRegionServerWithRetries throws 
RetriesExhaustedException with error message

{code}
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to 
contact region server null for region , row '----xxx', 
but failed after 10 attempts.
Exceptions:
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1008)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:546)
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4893) HConnectionImplementation closed-but-not-deleted, need a way to find the state of connection

2011-11-29 Thread Mubarak Seyed (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mubarak Seyed updated HBASE-4893:
-

Description: 
In abort() of HConnectionManager$HConnectionImplementation, instance of 
HConnectionImplementation is marked as this.closed=true.

There is no way for client application to check the hbase client connection 
whether it is still opened/good (this.closed=false) or not. We need a method to 
validate the state of a connection like isClosed().

{code}
public boolean isClosed(){
   return this.closed;
} 
{code}

Once the connection is closed and it should get deleted. Client application 
still gets a connection from HConnectionManager.getConnection(Configuration) 
and tries to make a RPC call to RS, since connection is already closed, 
HConnectionImplementation.getRegionServerWithRetries throws 
RetriesExhaustedException with error message

{code}
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to 
contact region server null for region , row '----xxx', 
but failed after 10 attempts.
Exceptions:
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1008)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:546)
{code}

  was:
In abort() of HConnectionManager$HConnectionImplementation, instance of 
HConnectionImplementation is marked as this.closed=true.

There is no way for client application to check the hbase client connection 
whether it is still opened/good (this.closed=false) or not. We need a method to 
validate the state of a connection like isClosed().

{code}
public boolean isClosed(){
   return this.closed;
} 
{code}

Once the connection is closed and it is not deleted. Client application still 
gets a connection from HConnectionManager.getConnection(Configuration) and 
tries to make a RPC call to RS, since connection is already closed, 
HConnectionImplementation.getRegionServerWithRetries throws 
RetriesExhaustedException with error message

{code}
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to 
contact region server null for region , row '----xxx', 
but failed after 10 attempts.
Exceptions:
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
java.io.IOException: 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
 closed
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1008)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:546)
{code}


 HConnectionImplementation closed-but-not-deleted, need a way to find the 
 state of connection

[jira] [Commented] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change

2011-11-29 Thread Todd Lipcon (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159487#comment-13159487
 ] 

Todd Lipcon commented on HBASE-4820:


I'll commit this at the end of the day if there are no objections.

 Distributed log splitting coding enhancement to make it easier to understand, 
 no semantics change
 -

 Key: HBASE-4820
 URL: https://issues.apache.org/jira/browse/HBASE-4820
 Project: HBase
  Issue Type: Improvement
  Components: wal
Affects Versions: 0.92.0, 0.94.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
  Labels: newbie
 Fix For: 0.92.0, 0.94.0

 Attachments: 
 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch,
  
 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch,
  0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, 
 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, 
 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch


 In reviewing distributed log splitting feature, we found some cosmetic 
 issues.  They make the code hard to understand.
 It will be great to fix them.  For this issue, there should be no semantic 
 change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-627) Disable table doesn't work reliably

2011-11-29 Thread Jean-Daniel Cryans (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159489#comment-13159489
 ] 

Jean-Daniel Cryans commented on HBASE-627:
--

@Erwin

This jira was resolved 3 years ago, if you have an issue would you mind opening 
a new jira?

 Disable table doesn't work reliably
 ---

 Key: HBASE-627
 URL: https://issues.apache.org/jira/browse/HBASE-627
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.2.0
 Environment: Hadoop/HBase on two nodes
Reporter: Michaela Buergle
Assignee: Jim Kellerman
Priority: Critical
 Fix For: 0.2.0

 Attachments: disableTable31.log, disableTable5.log, patch.txt, 
 patch.txt


 When creating a couple of tables like this:
 1) create an empty table
 2) disable table, add new column family, enable table
 3) put 100 small documents into newly created column
 around once in 10 tries the disable doesn't happen.
 I have no clue as to why the table isn't disabled in the first place, but if 
 this occurs, two things in HBaseAdmin.disableTable() strike me as odd:
 - after numRetries tries to wait for disabling we exit the loop; there is no 
 exception or error message:
 ...
 2008-05-14 16:19:47,903 INFO org.apache.hadoop.hbase.client.HBaseAdmin: 
 Disabled table table31
 2008-05-14 16:19:47,910 INFO org.apache.hadoop.ipc.Server: IPC Server handler 
 3 on 6, call addColumn(table31, {name: document, max versions: 3, 
 compression: NONE, in memory: false, block cache enabled: false, max length: 
 2147483647, time to live: FOREVER, bloom filter: none}) from 
 XXX.XX.40.36:47116: error: org.apache.hadoop.hbase.TableNotDisabledException: 
 table31
 ...
 - the scanner iterates over HRegionInfos of several tables. If any one of 
 those is disabled, we also leave the loop as if the requested table had been 
 disabled.
 I've had this disabling problem occur quite reliably over the last days - 
 today I couldn't reproduce it, though HBase version hasn't changed. ???

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4888) Not close ResultScanner cause the cluster abnormal ( RS memory increase)

2011-11-29 Thread Todd Lipcon (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159497#comment-13159497
 ] 

Todd Lipcon commented on HBASE-4888:


Something broken about our reseek stuff?

 Not close ResultScanner cause the cluster abnormal ( RS memory increase)
 

 Key: HBASE-4888
 URL: https://issues.apache.org/jira/browse/HBASE-4888
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.3
 Environment: CentOS 5.5 final, 
 hadoop-0.20.2-cdh3u1,hbase-0.20.2-cdh3u1
Reporter: Yuan Kang
  Labels: ResultScanner
   Original Estimate: 672h
  Remaining Estimate: 672h

 A ResultScanner is created in client side, If the user doesn't invoke the 
 ResultScanner.close() ,it happens that the memory of RegionServer increase 
 rapidly and hold for a long time. Finally,the cluster goes to an abnormal 
 status
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4605) Constraints

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159508#comment-13159508
 ] 

jirapos...@reviews.apache.org commented on HBASE-4605:
--



bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   Most of what I see is just style issues, and a number of the 
HTableDescriptor changes seem unnecessary.  If we clean up those, and assuming 
the latest patch has some documentation, I think this is looking pretty good.

Uploading the latest (with some of your recommended changes) after this. Couple 
of concerns may still need to be worked out around IntegerConstraint and the 
mods to HTD. Let me know what you think.


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 
22
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59947#file59947line22
bq.  
bq.   This differs from what seems to be the latest patch in JIRA 
(java_HBASE-4605_v3.txt).  Are there other differences than the javadoc?
bq.   
bq.   For a patch of this size, review still belongs here in review board 
(even with HadoopQA patch testing), so we should keep both versions in sync.  
Please update the diff here with the latest patch.

Yeah, this is the old version. Whats up on the ticket is just documentation. 
I'll put up the latest here after making the changes.


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 
250
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line250
bq.  
bq.   if / else bodies should be contained in braces.

done.


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 
233
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line233
bq.  
bq.   Be consistent with braces here

Just following the requirements of java :) Done.


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 
22
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line22
bq.  
bq.   I see the usage outside of HTD now, but still don't see where this 
provides more utility than Bytes.toString()

By pulling the functionality out of the HTD we can be sure that outside classes 
don't have to worry about making sure they use the 'right' (de)serialization of 
they keys. Right now we don't want to to anything more than just 
Bytes.toString(), but in the future we could make it something more complex. 
For example, take the case where user specified keys are prepended with a 
special identifier, so we can tell the difference between the two. 
Alternatively, we could do some cool stuff to cut down on object creation 
(potentially) with the key conversion or make it faster or more compact or w/e. 


Basically, I'm just adding the encapsulation so classes outside of the HTD 
don't have to know about the fact that the HTD is using Bytes.toString() (and 
encapsulation is good, right?). This way it is obvious, to an outside class, 
what is going - oh, it just serializing the key. This makes the addCP code much 
cleaner too. 


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 
86
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line86
bq.  
bq.   style nit: for loop body should be enclosed in braces

Done


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 
76
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line76
bq.  
bq.   Add constraints.size() to this message and you don't need the 
separate LOG.debug() message.

Done


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 
61
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line61
bq.  
bq.   style nit: if body should be contained in braces

Done.


bq.  On 2011-11-29 08:36:29, Gary Helmling wrote:
bq.   src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, 
line 32
bq.   https://reviews.apache.org/r/2579/diff/6/?file=59946#file59946line32
bq.  
bq.   This still seems odd to me.  I can't see storing an integer value 
encoded as a string, in order to be able to check for it being a valid integer.
bq.   
bq.   I would think most users would be storing integer values using 
Bytes.toInt(), for which the checks here would not work.

The only problem with using the Bytes deserialization checking is that even if 
you put a string in as bytes, it can be read out as an integer (in certain 
cases). Therefore, you could get a case where something that

[jira] [Commented] (HBASE-4867) A tool to merge configuration files

2011-11-29 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159509#comment-13159509
 ] 

Phabricator commented on HBASE-4867:


todd has commented on the revision [jira] [HBASE-4867] A tool to merge 
configuration files.

  It seems it would be preferable to write this in Java rather than Python, 
since nothing else in HBase is written in python. But, failing a complete 
rewrite, I'd recommend you drop use of the with statement so it works in 
py2.4/RHEL5. Should be a trivial change to try..finally

INLINE COMMENTS
  src/main/python/hbase/merge_conf.py:8 This isn't compatible with Python 2.4, 
which is the latest version on RHEL5. Most of our customers still run RHEL5, 
rather than RHEL6, so this script wouldn't be compatible.

REVISION DETAIL
  https://reviews.facebook.net/D537


 A tool to merge configuration files
 ---

 Key: HBASE-4867
 URL: https://issues.apache.org/jira/browse/HBASE-4867
 Project: HBase
  Issue Type: New Feature
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D537.1.patch


 With our cluster configuration setup it would be good to have a tool that 
 would merge HBase configuration, so that files appearing later in the list 
 would override properties specified in earlier files. This way we could merge 
 application-specific configuration file with the cluster-specific 
 configuration file (with the latter overriding the former) and produce a 
 single HBase configuration file to install on the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4867) A tool to merge configuration files

2011-11-29 Thread Phabricator (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-4867:
---

Attachment: D537.2.patch

mbautin updated the revision [jira] [HBASE-4867] A tool to merge configuration 
files.
Reviewers: todd, Karthik, tedyu, stack, JIRA

  Not breaking long words (e.g. URLs) that might appear in the description.

REVISION DETAIL
  https://reviews.facebook.net/D537

AFFECTED FILES
  src/main/python/hbase/merge_conf.py


 A tool to merge configuration files
 ---

 Key: HBASE-4867
 URL: https://issues.apache.org/jira/browse/HBASE-4867
 Project: HBase
  Issue Type: New Feature
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D537.1.patch, D537.2.patch


 With our cluster configuration setup it would be good to have a tool that 
 would merge HBase configuration, so that files appearing later in the list 
 would override properties specified in earlier files. This way we could merge 
 application-specific configuration file with the cluster-specific 
 configuration file (with the latter overriding the former) and produce a 
 single HBase configuration file to install on the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4605) Constraints

2011-11-29 Thread Jesse Yates (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159513#comment-13159513
 ] 

Jesse Yates commented on HBASE-4605:


Latest patch does _not_ have labeling of tests since I wanted to wait and see 
what dev@ thinks about the latest testing stuff.

 Constraints
 ---

 Key: HBASE-4605
 URL: https://issues.apache.org/jira/browse/HBASE-4605
 Project: HBase
  Issue Type: Improvement
  Components: client, coprocessors
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, 
 java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch


 From Jesse's comment on dev:
 {quote}
 What I would like to propose is a simple interface that people can use to 
 implement a 'constraint' (matching the classic database definition). This 
 would help ease of adoption by helping HBase more easily check that box, help 
 minimize code duplication across organizations, and lead to easier adoption.
 Essentially, people would implement a 'Constraint' interface for checking 
 keys before they are put into a table. Puts that are valid get written to the 
 table, but if not people can will throw an exception that gets propagated 
 back to the client explaining why the put was invalid.
 Constraints would be set on a per-table basis and the user would be expected 
 to ensure the jars containing the constraint are present on the machines 
 serving that table.
 Yes, people could roll their own mechanism for doing this via coprocessors 
 each time, but this would make it easier to do so, so you only have to 
 implement a very minimal interface and not worry about the specifics.
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4867) A tool to merge configuration files

2011-11-29 Thread Phabricator (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-4867:
---

Attachment: D537.3.patch

mbautin updated the revision [jira] [HBASE-4867] A tool to merge configuration 
files.
Reviewers: todd, Karthik, tedyu, stack, JIRA

  Addressing Todd's comments. Tested with python 2.4.

REVISION DETAIL
  https://reviews.facebook.net/D537

AFFECTED FILES
  src/main/python/hbase/merge_conf.py


 A tool to merge configuration files
 ---

 Key: HBASE-4867
 URL: https://issues.apache.org/jira/browse/HBASE-4867
 Project: HBase
  Issue Type: New Feature
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D537.1.patch, D537.2.patch, D537.3.patch


 With our cluster configuration setup it would be good to have a tool that 
 would merge HBase configuration, so that files appearing later in the list 
 would override properties specified in earlier files. This way we could merge 
 application-specific configuration file with the cluster-specific 
 configuration file (with the latter overriding the former) and produce a 
 single HBase configuration file to install on the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4605) Constraints

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159516#comment-13159516
 ] 

jirapos...@reviews.apache.org commented on HBASE-4605:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
---

(Updated 2011-11-29 20:19:41.250509)


Review request for hbase.


Changes
---

Updating to incorporate Gary and Ted's latest (as of 11/29) comments.

Mostly cleanup in syntax. Also pushing up latest documentation changes that we 
previously on the Jira.


Summary
---

Most of the implementation for adding constraints as a coprocessor. 

Looking for general comments on style/structure, though nitpicks are ok too. 

Currently missing implementation for disableConstraints() since that will 
require adding removeCoprocessor() to HTD (also comments on if this is worth it 
would be good). 


This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605


Diffs (updated)
-

  src/docbkx/book.xml 3c12169 
  src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a 
  src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java 
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/constraint/package-info.java 
PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java 
PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java 
PRE-CREATION 
  
src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java
 PRE-CREATION 
  
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java 
PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java 
PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java 
PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java 
PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/2579/diff


Testing
---

Adding IntegrationTestConstraint and unit tests for Constraints and 
IntegerConstraint. All of those pass.


Thanks,

Jesse



 Constraints
 ---

 Key: HBASE-4605
 URL: https://issues.apache.org/jira/browse/HBASE-4605
 Project: HBase
  Issue Type: Improvement
  Components: client, coprocessors
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
 Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, 
 java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch


 From Jesse's comment on dev:
 {quote}
 What I would like to propose is a simple interface that people can use to 
 implement a 'constraint' (matching the classic database definition). This 
 would help ease of adoption by helping HBase more easily check that box, help 
 minimize code duplication across organizations, and lead to easier adoption.
 Essentially, people would implement a 'Constraint' interface for checking 
 keys before they are put into a table. Puts that are valid get written to the 
 table, but if not people can will throw an exception that gets propagated 
 back to the client explaining why the put was invalid.
 Constraints would be set on a per-table basis and the user would be expected 
 to ensure the jars containing the constraint are present on the machines 
 serving that table.
 Yes, people could roll their own mechanism for doing this via coprocessors 
 each time, but this would make it easier to do so, so you only have to 
 implement a very minimal interface and not worry about the specifics.
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4889) HRegionInfo.isMetaTable() should be true for -ROOT- regions

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159517#comment-13159517
 ] 

Hudson commented on HBASE-4889:
---

Integrated in HBase-TRUNK #2498 (See 
[https://builds.apache.org/job/HBase-TRUNK/2498/])
HBASE-4889 HRegionInfo.isMetaTable() should be true for -ROOT- regions

stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java


 HRegionInfo.isMetaTable() should be true for -ROOT- regions
 ---

 Key: HBASE-4889
 URL: https://issues.apache.org/jira/browse/HBASE-4889
 Project: HBase
  Issue Type: Bug
Reporter: Daniel Gómez Ferro
 Fix For: 0.92.0

 Attachments: HBASE-4889.patch


 According to its javadoc, HRegionInfo.isMetaTable() should return true if the 
 region is either a .META. or -ROOT- region, but only does so for .META. 
 regions.
 The change in behavior happened in HBASE-451

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4721) Retain Delete Markers after Major Compaction

2011-11-29 Thread Karthik Ranganathan (Resolved) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Ranganathan resolved HBASE-4721.


Resolution: Fixed

 Retain Delete Markers after Major Compaction
 

 Key: HBASE-4721
 URL: https://issues.apache.org/jira/browse/HBASE-4721
 Project: HBase
  Issue Type: New Feature
Reporter: Prakash Khemani
Assignee: Prakash Khemani

 There is a need to provide long TTLs for delete markers. This is useful when 
 replicating hbase logs from one cluster to another. The receiving cluster 
 shouldn't compact away the delete markers because the affected key-values 
 might still be on the way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4867) A tool to merge configuration files

2011-11-29 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159525#comment-13159525
 ] 

Phabricator commented on HBASE-4867:


todd has commented on the revision [jira] [HBASE-4867] A tool to merge 
configuration files.

  +1. (can't find the button to accept the diff)

REVISION DETAIL
  https://reviews.facebook.net/D537


 A tool to merge configuration files
 ---

 Key: HBASE-4867
 URL: https://issues.apache.org/jira/browse/HBASE-4867
 Project: HBase
  Issue Type: New Feature
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D537.1.patch, D537.2.patch, D537.3.patch


 With our cluster configuration setup it would be good to have a tool that 
 would merge HBase configuration, so that files appearing later in the list 
 would override properties specified in earlier files. This way we could merge 
 application-specific configuration file with the cluster-specific 
 configuration file (with the latter overriding the former) and produce a 
 single HBase configuration file to install on the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)

2011-11-29 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159531#comment-13159531
 ] 

Phabricator commented on HBASE-4218:


todd has commented on the revision [jira] [HBASE-4218] Delta encoding for keys 
in HFile.

  I only got through a little bit of the giant patch, but it looks well done 
and decently unit-tested, so I'm +1 once you have some cluster testing results 
that show it basically works :)

  Test-plan should include an upgrade test from an unpatched HFile v2 format 
and an HFile v1 (0.90) upgrade

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java:99 seems odd 
that the type of this is boolean whereas the IN_CACHE one is an Algorithm type. 
If it's a requirement that the algo be the same, then maybe rename this one to 
be DEFAULT_DELTA_ENCODING_IN_MEMORY_ENABLED
  src/main/java/org/apache/hadoop/hbase/KeyValue.java:2022 This interface name 
isn't quite clear to me, since it doesn't compare prefixes. Maybe 
SuffixComparator? Or ComparatorAssumingEqualPrefix (though that's a bit 
lengthy)?
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BitsetKeyDeltaEncoder.java:34-42
 should use inline HTML to format this right
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BitsetKeyDeltaEncoder.java:56
 s/writeHere/out/g for consistent style
  
src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BitsetKeyDeltaEncoder.java:69
 s/source/in/g
  src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoder.java:32-35 
use HTML ul...
  src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java:90 
typo
  src/main/java/org/apache/hadoop/hbase/io/hfile/EmptyBlockDeltaEncoder.java:29 
maybe NoOpDeltaEncoder is a better name? (it's not that the block is empty)

REVISION DETAIL
  https://reviews.facebook.net/D447


 Delta Encoding of KeyValues  (aka prefix compression)
 -

 Key: HBASE-4218
 URL: https://issues.apache.org/jira/browse/HBASE-4218
 Project: HBase
  Issue Type: Improvement
  Components: io
Affects Versions: 0.94.0
Reporter: Jacek Migdal
Assignee: Mikhail Bautin
  Labels: compression
 Attachments: D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, 
 D447.5.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff


 A compression for keys. Keys are sorted in HFile and they are usually very 
 similar. Because of that, it is possible to design better compression than 
 general purpose algorithms,
 It is an additional step designed to be used in memory. It aims to save 
 memory in cache as well as speeding seeks within HFileBlocks. It should 
 improve performance a lot, if key lengths are larger than value lengths. For 
 example, it makes a lot of sense to use it when value is a counter.
 Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) 
 shows that I could achieve decent level of compression:
  key compression ratio: 92%
  total compression ratio: 85%
  LZO on the same data: 85%
  LZO after delta encoding: 91%
 While having much better performance (20-80% faster decompression ratio than 
 LZO). Moreover, it should allow far more efficient seeking which should 
 improve performance a bit.
 It seems that a simple compression algorithms are good enough. Most of the 
 savings are due to prefix compression, int128 encoding, timestamp diffs and 
 bitfields to avoid duplication. That way, comparisons of compressed data can 
 be much faster than a byte comparator (thanks to prefix compression and 
 bitfields).
 In order to implement it in HBase two important changes in design will be 
 needed:
 -solidify interface to HFileBlock / HFileReader Scanner to provide seeking 
 and iterating; access to uncompressed buffer in HFileBlock will have bad 
 performance
 -extend comparators to support comparison assuming that N first bytes are 
 equal (or some fields are equal)
 Link to a discussion about something similar:
 http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windowssubj=Re+prefix+compression

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4893) HConnectionImplementation closed-but-not-deleted, need a way to find the state of connection

2011-11-29 Thread stack (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159533#comment-13159533
 ] 

stack commented on HBASE-4893:
--

HBASE-4805 added HConnection.isClosed.  HBASE-4805 was committed to 0.92 and to 
TRUNK.  Does that work for you Mubarak?  If so, can we close this issue as 
duplicate?  Thanks.

 HConnectionImplementation closed-but-not-deleted, need a way to find the 
 state of connection
 

 Key: HBASE-4893
 URL: https://issues.apache.org/jira/browse/HBASE-4893
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.1, 0.90.2, 0.90.3, 0.90.4
 Environment: Linux 2.6, HBase-0.90.1
Reporter: Mubarak Seyed
  Labels: hbase-client
 Fix For: 0.90.5


 In abort() of HConnectionManager$HConnectionImplementation, instance of 
 HConnectionImplementation is marked as this.closed=true.
 There is no way for client application to check the hbase client connection 
 whether it is still opened/good (this.closed=false) or not. We need a method 
 to validate the state of a connection like isClosed().
 {code}
 public boolean isClosed(){
return this.closed;
 } 
 {code}
 Once the connection is closed and it should get deleted. Client application 
 still gets a connection from HConnectionManager.getConnection(Configuration) 
 and tries to make a RPC call to RS, since connection is already closed, 
 HConnectionImplementation.getRegionServerWithRetries throws 
 RetriesExhaustedException with error message
 {code}
 Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying 
 to contact region server null for region , row 
 '----xxx', but failed after 10 attempts.
 Exceptions:
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1008)
   at org.apache.hadoop.hbase.client.HTable.get(HTable.java:546)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-4791) Allow Secure Zookeeper JAAS configuration to be programmatically set (rather than only by reading JAAS configuration file)

2011-11-29 Thread Eugene Koontz (Assigned) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koontz reassigned HBASE-4791:


Assignee: Eugene Koontz

 Allow Secure Zookeeper JAAS configuration to be programmatically set (rather 
 than only by reading JAAS configuration file)
 --

 Key: HBASE-4791
 URL: https://issues.apache.org/jira/browse/HBASE-4791
 Project: HBase
  Issue Type: Bug
Reporter: Eugene Koontz
Assignee: Eugene Koontz
  Labels: security, zookeeper

 In the currently proposed fix for HBASE-2418, there must be a JAAS file 
 specified in System.setProperty(java.security.auth.login.config). 
 However, it might be preferable to construct a JAAS configuration 
 programmatically, as is done with secure Hadoop (see 
 https://github.com/apache/hadoop-common/blob/a48eceb62c9b5c1a5d71ee2945d9eea2ed62527b/src/java/org/apache/hadoop/security/UserGroupInformation.java#L175).
 This would have the benefit of avoiding a usage of a system property setting, 
 and allow instead an HBase-local configuration setting. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4791) Allow Secure Zookeeper JAAS configuration to be programmatically set (rather than only by reading JAAS configuration file)

2011-11-29 Thread Eugene Koontz (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koontz updated HBASE-4791:
-

Issue Type: Improvement  (was: Bug)

 Allow Secure Zookeeper JAAS configuration to be programmatically set (rather 
 than only by reading JAAS configuration file)
 --

 Key: HBASE-4791
 URL: https://issues.apache.org/jira/browse/HBASE-4791
 Project: HBase
  Issue Type: Improvement
Reporter: Eugene Koontz
Assignee: Eugene Koontz
  Labels: security, zookeeper

 In the currently proposed fix for HBASE-2418, there must be a JAAS file 
 specified in System.setProperty(java.security.auth.login.config). 
 However, it might be preferable to construct a JAAS configuration 
 programmatically, as is done with secure Hadoop (see 
 https://github.com/apache/hadoop-common/blob/a48eceb62c9b5c1a5d71ee2945d9eea2ed62527b/src/java/org/apache/hadoop/security/UserGroupInformation.java#L175).
 This would have the benefit of avoiding a usage of a system property setting, 
 and allow instead an HBase-local configuration setting. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4894) Remove sbin from tgz made building 0.92.

2011-11-29 Thread stack (Created) (JIRA) 
Remove sbin from tgz made building 0.92.


 Key: HBASE-4894
 URL: https://issues.apache.org/jira/browse/HBASE-4894
 Project: HBase
  Issue Type: Bug
Reporter: stack


When I undo the tgz, I see we have an sbin dir with update-hbase-env.sh in it.  
Lets do this messing in 0.94.  Its incomplete story in 0.92.  Removing sbin dir 
from tgz.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4894) Remove sbin from tgz made building 0.92.

2011-11-29 Thread stack (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4894:
-

Attachment: 4894.txt

Just remove the include from assembly.

 Remove sbin from tgz made building 0.92.
 

 Key: HBASE-4894
 URL: https://issues.apache.org/jira/browse/HBASE-4894
 Project: HBase
  Issue Type: Bug
Reporter: stack
 Fix For: 0.92.0

 Attachments: 4894.txt


 When I undo the tgz, I see we have an sbin dir with update-hbase-env.sh in 
 it.  Lets do this messing in 0.94.  Its incomplete story in 0.92.  Removing 
 sbin dir from tgz.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4894) Remove sbin from tgz made building 0.92.

2011-11-29 Thread stack (Resolved) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-4894.
--

   Resolution: Fixed
Fix Version/s: 0.92.0
 Assignee: stack

Applied to branch 0.92

 Remove sbin from tgz made building 0.92.
 

 Key: HBASE-4894
 URL: https://issues.apache.org/jira/browse/HBASE-4894
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.92.0

 Attachments: 4894.txt


 When I undo the tgz, I see we have an sbin dir with update-hbase-env.sh in 
 it.  Lets do this messing in 0.94.  Its incomplete story in 0.92.  Removing 
 sbin dir from tgz.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0002-Verify-start-and-end-key-are-contained-in-the-encode.patch, 
 0003-Moved-to-a-uuid-tablename.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: 0003-Moved-to-a-uuid-tablename.patch

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0002-Verify-start-and-end-key-are-contained-in-the-encode.patch, 
 0003-Moved-to-a-uuid-tablename.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: 0002-Verify-start-and-end-key-are-contained-in-the-encode.patch

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0002-Verify-start-and-end-key-are-contained-in-the-encode.patch, 
 0003-Moved-to-a-uuid-tablename.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159582#comment-13159582
 ] 

Alex Newman commented on HBASE-4616:


I figured I would post what I have to get feedback. There are still some things 
to complete.

- Meta migration
- Fixing the coprocessor tests
- Add additional unit tests

But i figured it's better than subbing.

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0002-Verify-start-and-end-key-are-contained-in-the-encode.patch, 
 0003-Moved-to-a-uuid-tablename.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: (was: 0004-added-testSplit.patch)

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0002-Verify-start-and-end-key-are-contained-in-the-encode.patch, 
 0003-Moved-to-a-uuid-tablename.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4895) Change tablename format in meta to be the UUID of the tablename rather than the tablename.

2011-11-29 Thread Alex Newman (Created) (JIRA) 
Change tablename format in meta to be the UUID of the tablename rather than the 
tablename.
--

 Key: HBASE-4895
 URL: https://issues.apache.org/jira/browse/HBASE-4895
 Project: HBase
  Issue Type: Sub-task
Reporter: Alex Newman
Assignee: Alex Newman


This is something stack and I discussed at hadoop world. Overall I think it 
cleans thing up significantly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: 0004-added-testSplit.patch

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0002-Verify-start-and-end-key-are-contained-in-the-encode.patch, 
 0003-Moved-to-a-uuid-tablename.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4896) Add Phabricator + ARC Installation to Help Doc

2011-11-29 Thread Nicolas Spiegelberg (Created) (JIRA) 
Add Phabricator + ARC Installation to Help Doc
--

 Key: HBASE-4896
 URL: https://issues.apache.org/jira/browse/HBASE-4896
 Project: HBase
  Issue Type: Task
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Priority: Trivial


Need to add help documentation to discuss the basic way of interoperating with 
Phabricator (what it is, submitting diffs, etc).  Also should add installation 
instructions for using the ARC CLI tool.  ARC documentation is mostly completed 
here for the Hive project:

https://cwiki.apache.org/confluence/display/Hive/PhabricatorCodeReview

The main difference is that you should run 'mvn initialize -Darc' instead of 
'ant arc-setup'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4893) HConnectionImplementation closed-but-not-deleted, need a way to find the state of connection

2011-11-29 Thread Mubarak Seyed (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159605#comment-13159605
 ] 

Mubarak Seyed commented on HBASE-4893:
--

Stack,

How about delete the HConnectionImplementation instance from HBASE_INSTANCES 
map if HConnectionImplementation instance is in closed state. When application 
uses connection pool and if the HConnection is closed, it can check the 
isClosed() and tries to create a new HTable(), which in turn gets the 
connection from HConnectionManager.getConnection(conf), which is a stale 
connection (meaning the connection state is closed).

In abort() of HConnectionManager$HConnectionImplementation, when we mark 
this.closed=true, can't we clean-up the HConnectionImplementation instance for 
a configuration in HBASE_INSTANCES map so that 
HConnectionManager.getConnection(conf) creates a new instance of 
HConnectionImplementation with new ZK connection (to ZK ensemble).

Thanks,
Mubarak

 HConnectionImplementation closed-but-not-deleted, need a way to find the 
 state of connection
 

 Key: HBASE-4893
 URL: https://issues.apache.org/jira/browse/HBASE-4893
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.1, 0.90.2, 0.90.3, 0.90.4
 Environment: Linux 2.6, HBase-0.90.1
Reporter: Mubarak Seyed
  Labels: hbase-client
 Fix For: 0.90.5


 In abort() of HConnectionManager$HConnectionImplementation, instance of 
 HConnectionImplementation is marked as this.closed=true.
 There is no way for client application to check the hbase client connection 
 whether it is still opened/good (this.closed=false) or not. We need a method 
 to validate the state of a connection like isClosed().
 {code}
 public boolean isClosed(){
return this.closed;
 } 
 {code}
 Once the connection is closed and it should get deleted. Client application 
 still gets a connection from HConnectionManager.getConnection(Configuration) 
 and tries to make a RPC call to RS, since connection is already closed, 
 HConnectionImplementation.getRegionServerWithRetries throws 
 RetriesExhaustedException with error message
 {code}
 Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying 
 to contact region server null for region , row 
 '----xxx', but failed after 10 attempts.
 Exceptions:
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1008)
   at org.apache.hadoop.hbase.client.HTable.get(HTable.java:546)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4382) Region encoded name is hash of tablename + start key + regionid (timestamp); should include end key when hashing.

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159607#comment-13159607
 ] 

jirapos...@reviews.apache.org commented on HBASE-4382:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2963/
---

Review request for hbase.


Summary
---

Seems odd that region encoded name is same for regions if made in same second 
with same start key tough their end keys are different. It can happen in unit 
test. Should mix in the end key when coming up w/ the region name encoded name.


This addresses bug hbase-4382.
https://issues.apache.org/jira/browse/hbase-4382


Diffs
-

  src/main/java/org/apache/hadoop/hbase/HConstants.java d22f50a 
  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 0c1fa3f 
  src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 1c49dc5 
  src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java e5e60a8 
  src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aa8512b 
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 6af1f82 
  src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java 4135e55 
  src/main/java/org/apache/hadoop/hbase/client/MetaSearchRow.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 
08b7de3 
  src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java bf85bc1 
  src/main/java/org/apache/hadoop/hbase/rest/model/TableRegionModel.java 
67e7a04 
  src/main/resources/hbase-default.xml 7059c60 
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 66d808f 
  src/test/java/org/apache/hadoop/hbase/TestKeyValue.java 7af4db4 
  src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 940d726 
  src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java 
b579b29 
  
src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
 49bfc5a 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java 
477e772 
  
src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
 24903f3 
  src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java 4a8bb69 
  src/test/java/org/apache/hadoop/hbase/rest/model/TestTableRegionModel.java 
60e0e41 
  src/test/ruby/hbase/admin_test.rb 0c2672b 

Diff: https://reviews.apache.org/r/2963/diff


Testing
---


Thanks,

Alex



 Region encoded name is hash of tablename + start key + regionid (timestamp); 
 should include end key when hashing.
 -

 Key: HBASE-4382
 URL: https://issues.apache.org/jira/browse/HBASE-4382
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: Alex Newman
  Labels: noob

 Seems odd that region encoded name is same for regions if made in same second 
 with same start key tough their end keys are different.  It can happen in 
 unit test.  Should mix in the end key when coming up w/ the region name 
 encoded name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4382) Region encoded name is hash of tablename + start key + regionid (timestamp); should include end key when hashing.

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159609#comment-13159609
 ] 

jirapos...@reviews.apache.org commented on HBASE-4382:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2963/#review3566
---



src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
https://reviews.apache.org/r/2963/#comment7984

I promise not to commit this.


- Alex


On 2011-11-29 22:52:32, Alex Newman wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2963/
bq.  ---
bq.  
bq.  (Updated 2011-11-29 22:52:32)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Seems odd that region encoded name is same for regions if made in same 
second with same start key tough their end keys are different. It can happen in 
unit test. Should mix in the end key when coming up w/ the region name encoded 
name.
bq.  
bq.  
bq.  This addresses bug hbase-4382.
bq.  https://issues.apache.org/jira/browse/hbase-4382
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/HConstants.java d22f50a 
bq.src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 0c1fa3f 
bq.src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 
1c49dc5 
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java e5e60a8 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aa8512b 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
6af1f82 
bq.src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java 4135e55 
bq.src/main/java/org/apache/hadoop/hbase/client/MetaSearchRow.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 
08b7de3 
bq.src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java bf85bc1 
bq.src/main/java/org/apache/hadoop/hbase/rest/model/TableRegionModel.java 
67e7a04 
bq.src/main/resources/hbase-default.xml 7059c60 
bq.src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 66d808f 
bq.src/test/java/org/apache/hadoop/hbase/TestKeyValue.java 7af4db4 
bq.src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 940d726 
bq.src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java 
b579b29 
bq.
src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
 49bfc5a 
bq.src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java 
477e772 
bq.
src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
 24903f3 
bq.src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java 
4a8bb69 
bq.
src/test/java/org/apache/hadoop/hbase/rest/model/TestTableRegionModel.java 
60e0e41 
bq.src/test/ruby/hbase/admin_test.rb 0c2672b 
bq.  
bq.  Diff: https://reviews.apache.org/r/2963/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Alex
bq.  
bq.



 Region encoded name is hash of tablename + start key + regionid (timestamp); 
 should include end key when hashing.
 -

 Key: HBASE-4382
 URL: https://issues.apache.org/jira/browse/HBASE-4382
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: Alex Newman
  Labels: noob

 Seems odd that region encoded name is same for regions if made in same second 
 with same start key tough their end keys are different.  It can happen in 
 unit test.  Should mix in the end key when coming up w/ the region name 
 encoded name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4895) Change tablename format in meta to be the UUID of the tablename rather than the tablename.

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159611#comment-13159611
 ] 

jirapos...@reviews.apache.org commented on HBASE-4895:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2965/
---

Review request for hbase.


Summary
---

The issue is we have to have a custom compareter for metakey/rootkey scanning 
to work. One of the reasons why this is required is that the tablenames are 
currently lexically sorted.


This addresses bug HBASE-4895.
https://issues.apache.org/jira/browse/HBASE-4895


Diffs
-

  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 0c1fa3f 

Diff: https://reviews.apache.org/r/2965/diff


Testing
---


Thanks,

Alex



 Change tablename format in meta to be the UUID of the tablename rather than 
 the tablename.
 --

 Key: HBASE-4895
 URL: https://issues.apache.org/jira/browse/HBASE-4895
 Project: HBase
  Issue Type: Sub-task
Reporter: Alex Newman
Assignee: Alex Newman

 This is something stack and I discussed at hadoop world. Overall I think it 
 cleans thing up significantly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159612#comment-13159612
 ] 

jirapos...@reviews.apache.org commented on HBASE-2600:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2965/
---

(Updated 2011-11-29 22:58:47.832063)


Review request for hbase.


Summary (updated)
---

The issue is we have to have a custom compareter for metakey/rootkey scanning 
to work. One of the reasons why this is required is that the tablenames are 
currently lexically sorted.


This addresses bug HBASE-2600.
https://issues.apache.org/jira/browse/HBASE-2600


Diffs
-

  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 0c1fa3f 

Diff: https://reviews.apache.org/r/2965/diff


Testing
---


Thanks,

Alex



 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: Alex Newman

 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4893) HConnectionImplementation closed-but-not-deleted, need a way to find the state of connection

2011-11-29 Thread Ted Yu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159613#comment-13159613
 ] 

Ted Yu commented on HBASE-4893:
---

HConnectionManager.deleteStaleConnection() can be utilized in the above 
proposal.

 HConnectionImplementation closed-but-not-deleted, need a way to find the 
 state of connection
 

 Key: HBASE-4893
 URL: https://issues.apache.org/jira/browse/HBASE-4893
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.1, 0.90.2, 0.90.3, 0.90.4
 Environment: Linux 2.6, HBase-0.90.1
Reporter: Mubarak Seyed
  Labels: hbase-client
 Fix For: 0.90.5


 In abort() of HConnectionManager$HConnectionImplementation, instance of 
 HConnectionImplementation is marked as this.closed=true.
 There is no way for client application to check the hbase client connection 
 whether it is still opened/good (this.closed=false) or not. We need a method 
 to validate the state of a connection like isClosed().
 {code}
 public boolean isClosed(){
return this.closed;
 } 
 {code}
 Once the connection is closed and it should get deleted. Client application 
 still gets a connection from HConnectionManager.getConnection(Configuration) 
 and tries to make a RPC call to RS, since connection is already closed, 
 HConnectionImplementation.getRegionServerWithRetries throws 
 RetriesExhaustedException with error message
 {code}
 Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying 
 to contact region server null for region , row 
 '----xxx', but failed after 10 attempts.
 Exceptions:
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1008)
   at org.apache.hadoop.hbase.client.HTable.get(HTable.java:546)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159615#comment-13159615
 ] 

jirapos...@reviews.apache.org commented on HBASE-2600:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2968/
---

Review request for hbase.


Summary
---

This is an idea that Ryan and I have been kicking around on and off for a while 
now.

If regionnames were made of tablename+endrow instead of tablename+startrow, 
then in the metatables, doing a search for the region that contains the wanted 
row, we'd just have to open a scanner using passed row and the first row found 
by the scan would be that of the region we need (If offlined parent, we'd have 
to scan to the next row).

If we redid the meta tables in this format, we'd be using an access that is 
natural to hbase, a scan as opposed to the perverse, expensive 
getClosestRowBefore we currently have that has to walk backward in meta finding 
a containing region.

This issue is about changing the way we name regions.

If we were using scans, prewarming client cache would be near costless (as 
opposed to what we'll currently have to do which is first a getClosestRowBefore 
and then a scan from the closestrowbefore forward).

Converting to the new method, we'd have to run a migration on startup changing 
the content in meta.

Up to this, the randomid component of a region name has been the timestamp of 
region creation. HBASE-2531 32-bit encoding of regionnames waaay too 
susceptible to hash clashes proposes changing the randomid so that it contains 
actual name of the directory in the filesystem that hosts the region. If we had 
this in place, I think it would help with the migration to this new way of 
doing the meta because as is, the region name in fs is a hash of regionname... 
changing the format of the regionname would mean we generate a different 
hash... so we'd need hbase-2531 to be in place before we could do this change.


This addresses bug hbase-2600.
https://issues.apache.org/jira/browse/hbase-2600


Diffs
-

  src/main/java/org/apache/hadoop/hbase/HConstants.java d22f50a 
  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 0c1fa3f 
  src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 1c49dc5 
  src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java e5e60a8 
  src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aa8512b 
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 6af1f82 
  src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java 4135e55 
  src/main/java/org/apache/hadoop/hbase/client/MetaSearchRow.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 
08b7de3 
  src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java bf85bc1 
  src/main/java/org/apache/hadoop/hbase/rest/model/TableRegionModel.java 
67e7a04 
  src/main/resources/hbase-default.xml 7059c60 
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 66d808f 
  src/test/java/org/apache/hadoop/hbase/TestKeyValue.java 7af4db4 
  src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 940d726 
  src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java 
b579b29 
  
src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
 49bfc5a 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java 
477e772 
  
src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
 24903f3 
  src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java 4a8bb69 
  src/test/java/org/apache/hadoop/hbase/rest/model/TestTableRegionModel.java 
60e0e41 
  src/test/ruby/hbase/admin_test.rb 0c2672b 

Diff: https://reviews.apache.org/r/2968/diff


Testing
---


Thanks,

Alex



 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: Alex Newman

 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

2011-11-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159616#comment-13159616
 ] 

jirapos...@reviews.apache.org commented on HBASE-2600:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2968/#review3568
---



src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
https://reviews.apache.org/r/2968/#comment7985

Please ignore me.


- Alex


On 2011-11-29 23:00:08, Alex Newman wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2968/
bq.  ---
bq.  
bq.  (Updated 2011-11-29 23:00:08)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is an idea that Ryan and I have been kicking around on and off for a 
while now.
bq.  
bq.  If regionnames were made of tablename+endrow instead of 
tablename+startrow, then in the metatables, doing a search for the region that 
contains the wanted row, we'd just have to open a scanner using passed row and 
the first row found by the scan would be that of the region we need (If 
offlined parent, we'd have to scan to the next row).
bq.  
bq.  If we redid the meta tables in this format, we'd be using an access that 
is natural to hbase, a scan as opposed to the perverse, expensive 
getClosestRowBefore we currently have that has to walk backward in meta finding 
a containing region.
bq.  
bq.  This issue is about changing the way we name regions.
bq.  
bq.  If we were using scans, prewarming client cache would be near costless (as 
opposed to what we'll currently have to do which is first a getClosestRowBefore 
and then a scan from the closestrowbefore forward).
bq.  
bq.  Converting to the new method, we'd have to run a migration on startup 
changing the content in meta.
bq.  
bq.  Up to this, the randomid component of a region name has been the timestamp 
of region creation. HBASE-2531 32-bit encoding of regionnames waaay 
too susceptible to hash clashes proposes changing the randomid so that it 
contains actual name of the directory in the filesystem that hosts the region. 
If we had this in place, I think it would help with the migration to this new 
way of doing the meta because as is, the region name in fs is a hash of 
regionname... changing the format of the regionname would mean we generate a 
different hash... so we'd need hbase-2531 to be in place before we could do 
this change.
bq.  
bq.  
bq.  This addresses bug hbase-2600.
bq.  https://issues.apache.org/jira/browse/hbase-2600
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/HConstants.java d22f50a 
bq.src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 0c1fa3f 
bq.src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 
1c49dc5 
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java e5e60a8 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aa8512b 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
6af1f82 
bq.src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java 4135e55 
bq.src/main/java/org/apache/hadoop/hbase/client/MetaSearchRow.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 
08b7de3 
bq.src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java bf85bc1 
bq.src/main/java/org/apache/hadoop/hbase/rest/model/TableRegionModel.java 
67e7a04 
bq.src/main/resources/hbase-default.xml 7059c60 
bq.src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 66d808f 
bq.src/test/java/org/apache/hadoop/hbase/TestKeyValue.java 7af4db4 
bq.src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 940d726 
bq.src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java 
b579b29 
bq.
src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
 49bfc5a 
bq.src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java 
477e772 
bq.
src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
 24903f3 
bq.src/test/java/org/apache/hadoop/hbase/rest/TestStatusResource.java 
4a8bb69 
bq.
src/test/java/org/apache/hadoop/hbase/rest/model/TestTableRegionModel.java 
60e0e41 
bq.src/test/ruby/hbase/admin_test.rb 0c2672b 
bq.  
bq.  Diff: https://reviews.apache.org/r/2968/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Alex
bq.  
bq.



 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid

[jira] [Updated] (HBASE-4893) HConnectionImplementation closed-but-not-deleted, need a way to find the state of connection

2011-11-29 Thread Mubarak Seyed (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mubarak Seyed updated HBASE-4893:
-

Fix Version/s: 0.90.1

 HConnectionImplementation closed-but-not-deleted, need a way to find the 
 state of connection
 

 Key: HBASE-4893
 URL: https://issues.apache.org/jira/browse/HBASE-4893
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.1, 0.90.2, 0.90.3, 0.90.4
 Environment: Linux 2.6, HBase-0.90.1
Reporter: Mubarak Seyed
  Labels: hbase-client
 Fix For: 0.90.1, 0.90.5


 In abort() of HConnectionManager$HConnectionImplementation, instance of 
 HConnectionImplementation is marked as this.closed=true.
 There is no way for client application to check the hbase client connection 
 whether it is still opened/good (this.closed=false) or not. We need a method 
 to validate the state of a connection like isClosed().
 {code}
 public boolean isClosed(){
return this.closed;
 } 
 {code}
 Once the connection is closed and it should get deleted. Client application 
 still gets a connection from HConnectionManager.getConnection(Configuration) 
 and tries to make a RPC call to RS, since connection is already closed, 
 HConnectionImplementation.getRegionServerWithRetries throws 
 RetriesExhaustedException with error message
 {code}
 Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying 
 to contact region server null for region , row 
 '----xxx', but failed after 10 attempts.
 Exceptions:
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7eab48a7
  closed
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1008)
   at org.apache.hadoop.hbase.client.HTable.get(HTable.java:546)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2418) add support for ZooKeeper authentication

2011-11-29 Thread Eugene Koontz (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159618#comment-13159618
 ] 

Eugene Koontz commented on HBASE-2418:
--

HBase clients can authenticate with Kerberos using {{kinit}} and connect to 
hbase (e.g. with hbase shell) with 
{{-Djava.security.auth.login.config=/path/to/client/jaas.conf}} , where this 
configuration file is:

{code}
Client {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=false
 useTicketCache=true
 doNotPrompt=true
 renewTGT=true;
};
{code}




 add support for ZooKeeper authentication
 

 Key: HBASE-2418
 URL: https://issues.apache.org/jira/browse/HBASE-2418
 Project: HBase
  Issue Type: Improvement
  Components: master, regionserver
Reporter: Patrick Hunt
Assignee: Eugene Koontz
Priority: Critical
  Labels: security, zookeeper
 Fix For: 0.92.0, 0.94.0

 Attachments: 2418.addendum, HBASE-2418-6.patch, HBASE-2418-6.patch


 Some users may run a ZooKeeper cluster in multi tenant mode meaning that 
 more than one client service would
 like to share a single ZooKeeper service instance (cluster). In this case the 
 client services typically want to protect
 their data (ZK znodes) from access by other services (tenants) on the 
 cluster. Say you are running HBase and Solr 
 and Neo4j, or multiple HBase instances, etc... having 
 authentication/authorization on the znodes is important for both 
 security and helping to ensure that services don't interact negatively (touch 
 each other's data).
 Today HBase does not have support for authentication or authorization. This 
 should be added to the HBase clients
 that are accessing the ZK cluster. In general it means calling addAuthInfo 
 once after a session is established:
 http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String,
  byte[])
 with a user specific credential, often times this is a shared secret or 
 certificate. You may be able to statically configure this
 in some cases (config string or file to read from), however in my case in 
 particular you may need to access it programmatically,
 which adds complexity as the end user may need to load code into HBase for 
 accessing the credential.
 Secondly you need to specify a non world ACL when interacting with znodes 
 (create primarily):
 http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html
 http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html
 Feel free to ping the ZooKeeper team if you have questions. It might also be 
 good to discuss with some 
 potential end users - in particular regarding how the end user can specify 
 the credential.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: (was: 
0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch)

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: (was: 0003-Moved-to-a-uuid-tablename.patch)

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: (was: 
0002-Verify-start-and-end-key-are-contained-in-the-encode.patch)

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Alex Newman (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4616:
---

Attachment: HBASE-4616.patch

I condensed the three patches into one

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: HBASE-4616.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4721) Retain Delete Markers after Major Compaction

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159669#comment-13159669
 ] 

Hudson commented on HBASE-4721:
---

Integrated in HBase-TRUNK #2500 (See 
[https://builds.apache.org/job/HBase-TRUNK/2500/])
HBASE-4721 Retain Delete Markers after Major Compaction

Summary: major compactions *try* to retain delete markers for the configured
time interval

Test Plan:
added a new test testDeleteMarkerLongevity() in TestStoreScanner

following tests passed
TestMemStore
TestStoreScanner
TestQueryMatcher
TestCompaction

Reviewers: jgray, dhruba, lhofhansl, Karthik, nspiegelberg

Reviewed By: nspiegelberg

CC: HBase Diffs Facebook Group, lhofhansl, khemani, nspiegelberg

Differential Revision: 321

karthik : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java


 Retain Delete Markers after Major Compaction
 

 Key: HBASE-4721
 URL: https://issues.apache.org/jira/browse/HBASE-4721
 Project: HBase
  Issue Type: New Feature
Reporter: Prakash Khemani
Assignee: Prakash Khemani

 There is a need to provide long TTLs for delete markers. This is useful when 
 replicating hbase logs from one cluster to another. The receiving cluster 
 shouldn't compact away the delete markers because the affected key-values 
 might still be on the way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4892) [book] book updates 11-29-2011

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159668#comment-13159668
 ] 

Hudson commented on HBASE-4892:
---

Integrated in HBase-TRUNK #2500 (See 
[https://builds.apache.org/job/HBase-TRUNK/2500/])
hbase-4892 book.xml, ops_mgt.xml book changes.


 [book] book updates 11-29-2011
 --

 Key: HBASE-4892
 URL: https://issues.apache.org/jira/browse/HBASE-4892
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4892.patch


 book.xml
 * MR chapter, added using HBase table as reducer  
 * data model, added info about different types of tombstones per conversation 
 this week about deletes.
 ops_mgt.xml
 * added region mgt section, mentioned major compation and region merges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Ted Yu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159676#comment-13159676
 ] 

Ted Yu commented on HBASE-4616:
---

Where is MetaSearchRow defined ?
What's the difference between MetaSearchRow.getStartSearchRow(tableName, null) 
and MetaSearchRow.getStartSearchRow(tableName, HConstants.EMPTY_BYTE_ARRAY) ?

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: HBASE-4616.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Ted Yu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159694#comment-13159694
 ] 

Ted Yu commented on HBASE-4616:
---

In HRegionInfo.java:
{code}
  public static final int END_OF_TABLE_TABLE_NAME = END_OF_TABLE_NAME + 1;
{code}
I think END_OF_TABLE_NAME_FOR_EMPTY_ENDKEY would be a better name.

In createRegionName():
{code}
if (id != null || id.length  0 ) {
{code}
 should be used above.

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: HBASE-4616.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-4729) Clash between region unassign and splitting kills the master

2011-11-29 Thread ramkrishna.s.vasudevan (Assigned) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reassigned HBASE-4729:
-

Assignee: stack  (was: ramkrishna.s.vasudevan)

Assigning to your name Stack.

 Clash between region unassign and splitting kills the master
 

 Key: HBASE-4729
 URL: https://issues.apache.org/jira/browse/HBASE-4729
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: stack
Priority: Critical
 Fix For: 0.92.0, 0.94.0

 Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729-v5.txt, 
 4729-v6-092.txt, 4729-v6-trunk.txt, 4729.txt


 I was running an online alter while regions were splitting, and suddenly the 
 master died and left my table half-altered (haven't restarted the master yet).
 What killed the master:
 {quote}
 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unexpected ZK exception creating node CLOSING
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
 at 
 org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
 at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
 at 
 org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
 at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {quote}
 A znode was created because the region server was splitting the region 4 
 seconds before:
 {quote}
 2011-11-02 17:06:40,704 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of 
 region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
 2011-11-02 17:06:40,704 DEBUG 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 regionserver:62023-0x132f043bbde0710 Creating ephemeral node for 
 f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Attempting to transition node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLITTING
 ...
 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:62023-0x132f043bbde0710 Successfully transitioned node 
 f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to 
 RS_ZK_REGION_SPLIT
 2011-11-02 17:06:44,061 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for f7e1783e65ea8d621a4bc96ad310f101
 {quote}
 Now that the master is dead the region server is spewing those last two lines 
 like mad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4721) Retain Delete Markers after Major Compaction

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159746#comment-13159746
 ] 

Hudson commented on HBASE-4721:
---

Integrated in HBase-TRUNK-security #15 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/15/])
HBASE-4721 Retain Delete Markers after Major Compaction

Summary: major compactions *try* to retain delete markers for the configured
time interval

Test Plan:
added a new test testDeleteMarkerLongevity() in TestStoreScanner

following tests passed
TestMemStore
TestStoreScanner
TestQueryMatcher
TestCompaction

Reviewers: jgray, dhruba, lhofhansl, Karthik, nspiegelberg

Reviewed By: nspiegelberg

CC: HBase Diffs Facebook Group, lhofhansl, khemani, nspiegelberg

Differential Revision: 321

karthik : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java


 Retain Delete Markers after Major Compaction
 

 Key: HBASE-4721
 URL: https://issues.apache.org/jira/browse/HBASE-4721
 Project: HBase
  Issue Type: New Feature
Reporter: Prakash Khemani
Assignee: Prakash Khemani

 There is a need to provide long TTLs for delete markers. This is useful when 
 replicating hbase logs from one cluster to another. The receiving cluster 
 shouldn't compact away the delete markers because the affected key-values 
 might still be on the way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4889) HRegionInfo.isMetaTable() should be true for -ROOT- regions

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159744#comment-13159744
 ] 

Hudson commented on HBASE-4889:
---

Integrated in HBase-TRUNK-security #15 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/15/])
HBASE-4889 HRegionInfo.isMetaTable() should be true for -ROOT- regions

stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java


 HRegionInfo.isMetaTable() should be true for -ROOT- regions
 ---

 Key: HBASE-4889
 URL: https://issues.apache.org/jira/browse/HBASE-4889
 Project: HBase
  Issue Type: Bug
Reporter: Daniel Gómez Ferro
 Fix For: 0.92.0

 Attachments: HBASE-4889.patch


 According to its javadoc, HRegionInfo.isMetaTable() should return true if the 
 region is either a .META. or -ROOT- region, but only does so for .META. 
 regions.
 The change in behavior happened in HBASE-451

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4892) [book] book updates 11-29-2011

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159745#comment-13159745
 ] 

Hudson commented on HBASE-4892:
---

Integrated in HBase-TRUNK-security #15 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/15/])
hbase-4892 book.xml, ops_mgt.xml book changes.


 [book] book updates 11-29-2011
 --

 Key: HBASE-4892
 URL: https://issues.apache.org/jira/browse/HBASE-4892
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4892.patch


 book.xml
 * MR chapter, added using HBase table as reducer  
 * data model, added info about different types of tombstones per conversation 
 this week about deletes.
 ops_mgt.xml
 * added region mgt section, mentioned major compation and region merges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4888) Not close ResultScanner cause the cluster abnormal ( RS memory increase)

2011-11-29 Thread Yuan Kang (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159750#comment-13159750
 ] 

Yuan Kang commented on HBASE-4888:
--

I came into this issus in online system and find some reference in 《HBase: The 
Definitive Guide》. 

It writes  Scanner Leases
Make sure you release a scanner instance as timely as possible. An open scanner 
holds quite a few resources on the server side, which could accumulate to a 
large amount of heap space occupied. When you are done with the current scan 
call close(), and consider adding this into a try/finally construct to ensure 
it is called, even if there are exceptions or errors during the iterations.

Like row locks, scanners are protected against stray clients blocking resources 
for too long, using the same lease based mechanisms. You need to set the same 
configuration property to modify the timeout threshold (in milliseconds):
property
namehbase.regionserver.lease.period/name
value12/value
/property
You need to make sure that the property is set to an appropriate value that 
make sense for locks and the scanner leases.


But I find 'hbase.regionserver.lease.period' don't work well and Our RS hold 
high memory for a much longer time more than 'hbase.regionserver.lease.period' 
.After I add 'rs.close()' in my code,it disappeared




 Not close ResultScanner cause the cluster abnormal ( RS memory increase)
 

 Key: HBASE-4888
 URL: https://issues.apache.org/jira/browse/HBASE-4888
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.3
 Environment: CentOS 5.5 final, 
 hadoop-0.20.2-cdh3u1,hbase-0.20.2-cdh3u1
Reporter: Yuan Kang
  Labels: ResultScanner
   Original Estimate: 672h
  Remaining Estimate: 672h

 A ResultScanner is created in client side, If the user doesn't invoke the 
 ResultScanner.close() ,it happens that the memory of RegionServer increase 
 rapidly and hold for a long time. Finally,the cluster goes to an abnormal 
 status
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4616) Update hregion encoded name to reduce logic and prevent region collisions in META

2011-11-29 Thread Ted Yu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159761#comment-13159761
 ] 

Ted Yu commented on HBASE-4616:
---

In SplitTransaction.java, getDaughterRegionIdTimestamp() is removed.
I think we should take care of clock skew.

Also, using hri.getRegionId() - 1 as Id for daughter region Ids is not 
intuitive.

 Update hregion encoded name to reduce logic and prevent region collisions in 
 META
 -

 Key: HBASE-4616
 URL: https://issues.apache.org/jira/browse/HBASE-4616
 Project: HBase
  Issue Type: Umbrella
Reporter: Alex Newman
Assignee: Alex Newman
 Attachments: HBASE-4616.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4897) [book] clarifying scan example, adding troubleshooting example

2011-11-29 Thread Doug Meil (Created) (JIRA) 
[book] clarifying scan example, adding troubleshooting example
--

 Key: HBASE-4897
 URL: https://issues.apache.org/jira/browse/HBASE-4897
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor


book.xml
* the scan example had a comment like start key != stop key.  Broke this into 
2 comments in the example, indicating that the start key was inclusive, and the 
stop key was exclusive

troubleshooting.xml
* adding you thought you were on the cluster, but you're actually local 
example in a new MapReduce section in the troubleshooting chapter.  Came up in 
dist-list recently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4897) [book] clarifying scan example, adding troubleshooting example

2011-11-29 Thread Doug Meil (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4897:
-

Status: Patch Available  (was: Open)

 [book] clarifying scan example, adding troubleshooting example
 --

 Key: HBASE-4897
 URL: https://issues.apache.org/jira/browse/HBASE-4897
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4897.patch


 book.xml
 * the scan example had a comment like start key != stop key.  Broke this 
 into 2 comments in the example, indicating that the start key was inclusive, 
 and the stop key was exclusive
 troubleshooting.xml
 * adding you thought you were on the cluster, but you're actually local 
 example in a new MapReduce section in the troubleshooting chapter.  Came up 
 in dist-list recently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4897) [book] clarifying scan example, adding troubleshooting example

2011-11-29 Thread Doug Meil (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4897:
-

Attachment: docbkx_HBASE_4897.patch

 [book] clarifying scan example, adding troubleshooting example
 --

 Key: HBASE-4897
 URL: https://issues.apache.org/jira/browse/HBASE-4897
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_HBASE_4897.patch


 book.xml
 * the scan example had a comment like start key != stop key.  Broke this 
 into 2 comments in the example, indicating that the start key was inclusive, 
 and the stop key was exclusive
 troubleshooting.xml
 * adding you thought you were on the cluster, but you're actually local 
 example in a new MapReduce section in the troubleshooting chapter.  Came up 
 in dist-list recently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4841) If I call split fast enough, while inserting, rows disappear.

2011-11-29 Thread Jean-Daniel Cryans (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159771#comment-13159771
 ] 

Jean-Daniel Cryans commented on HBASE-4841:
---

The test passes for me 100% of the time on both trunk and 0.92 when I wrap the 
admin.split in order to catch the region offline exception that it gets 
sometimes.

 If I call split fast enough, while inserting, rows disappear. 
 --

 Key: HBASE-4841
 URL: https://issues.apache.org/jira/browse/HBASE-4841
 Project: HBase
  Issue Type: Bug
Reporter: Alex Newman
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Attachments: 1, log, log2


 I'll attach a unit test for this. Basically if you call split, while 
 inserting data you can get to the point to where the cluster becomes 
 unstable, or rows will  disappear. The unit test gives you some flexibility 
 of:
 - How many rows
 - How wide the rows are
 - The frequency of the split. 
 The default settings crash unit tests or cause the unit tests to fail on my 
 laptop. On my macbook air, i could actually turn down the number of total 
 rows, and the frequency of the splits which is surprising. I think this is 
 because the macbook air has much better IO than my backup acer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-4898) [book] fixing minor MR summary bug, adding link to HBase-MR classpath info in Javadoc

2011-11-29 Thread Doug Meil (Created) (JIRA) 
[book] fixing minor MR summary bug, adding link to HBase-MR classpath info in 
Javadoc
-

 Key: HBASE-4898
 URL: https://issues.apache.org/jira/browse/HBASE-4898
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor


book.xml
* MapReduce.  Correcting a sentence in the summary with no reducer example 
trying to describe the need for a pre-existing target table.

troubleshooting.xml
* MapReduce.  Adding link to javadoc that has some classpath info. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4898) [book] fixing minor MR summary bug, adding link to HBase-MR classpath info in Javadoc

2011-11-29 Thread Doug Meil (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-4898:
-

Status: Patch Available  (was: Open)

 [book] fixing minor MR summary bug, adding link to HBase-MR classpath info in 
 Javadoc
 -

 Key: HBASE-4898
 URL: https://issues.apache.org/jira/browse/HBASE-4898
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_hbase_4898.patch


 book.xml
 * MapReduce.  Correcting a sentence in the summary with no reducer example 
 trying to describe the need for a pre-existing target table.
 troubleshooting.xml
 * MapReduce.  Adding link to javadoc that has some classpath info. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4789) On split, parent region is sticking around in oldest sequenceid to region map though not online; we don't cleanup WALs.

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159781#comment-13159781
 ] 

Hudson commented on HBASE-4789:
---

Integrated in HBase-0.92 #163 (See 
[https://builds.apache.org/job/HBase-0.92/163/])
HBASE-4853 HBASE-4789 does overzealous pruning of seqids

stack : 
Files : 
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java


 On split, parent region is sticking around in oldest sequenceid to region map 
 though not online; we don't cleanup WALs.
 ---

 Key: HBASE-4789
 URL: https://issues.apache.org/jira/browse/HBASE-4789
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 0.92.0

 Attachments: 4789-v2.txt, 4789-v3.txt, 4789-v4.txt, 4789.txt


 Here is log for a particular region:
 {code}
 2011-11-15 05:46:31,382 INFO 
 org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the 
 master to process the split for 8bbd7388262dc8cb1ce2cf4f04a7281d
 2011-11-15 05:46:31,483 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 regionserver:7003-0x1337b0b92cd000a-0x1337b0b92cd000a Attempting to 
 transition node 8bbd7388262dc8cb1ce2cf4f04a7281d from RS_ZK_REGION_SPLIT to 
 RS_ZK_REG
 ION_SPLIT
 2011-11-15 05:46:31,484 INFO 
 org.apache.hadoop.hbase.regionserver.SplitRequest: Region split, META 
 updated, and report to master. 
 Parent=TestTable,0862220095,1321335865649.8bbd7388262dc8cb1ce2cf4f04a7281d., 
 new regions: TestTab
 le,0862220095,1321335989689.f00c683df3182d8ef33e315f77ca539c., 
 TestTable,0892568091,1321335989689.a56ca1eff5b4401432fcba04b4e851f8.. Split 
 took 1sec
 2011-11-15 05:46:37,705 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
 Compacting 
 hdfs://sv4r11s38:7000/hbase/TestTable/a56ca1eff5b4401432fcba04b4e851f8/info/9ce16d8fa94e4938964c04775a6fa1a7.8bbd7388262dc8cb1ce2cf4f04a7281d-
 hdfs://sv4r11s38:7000/hbase/TestTable/8bbd7388262dc8cb1ce2cf4f04a7281d/info/9ce16d8fa94e4938964c04775a6fa1a7-top,
  keycount=717559, bloomtype=NONE, size=711.1m
 2011-11-15 05:46:37,705 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
 Compacting 
 hdfs://sv4r11s38:7000/hbase/TestTable/a56ca1eff5b4401432fcba04b4e851f8/info/9213f4d7ee9b4fda857a97603a001f9e.8bbd7388262dc8cb1ce2cf4f04a7281d-
 hdfs://sv4r11s38:7000/hbase/TestTable/8bbd7388262dc8cb1ce2cf4f04a7281d/info/9213f4d7ee9b4fda857a97603a001f9e-top,
  keycount=416691, bloomtype=NONE, size=412.9m
 2011-11-15 05:46:53,090 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
 Compacting 
 hdfs://sv4r11s38:7000/hbase/TestTable/f00c683df3182d8ef33e315f77ca539c/info/9ce16d8fa94e4938964c04775a6fa1a7.8bbd7388262dc8cb1ce2cf4f04a7281d-
 hdfs://sv4r11s38:7000/hbase/TestTable/8bbd7388262dc8cb1ce2cf4f04a7281d/info/9ce16d8fa94e4938964c04775a6fa1a7-bottom,
  keycount=717559, bloomtype=NONE, size=711.1m
 2011-11-15 05:46:53,090 DEBUG org.apache.hadoop.hbase.regionserver.Store: 
 Compacting 
 hdfs://sv4r11s38:7000/hbase/TestTable/f00c683df3182d8ef33e315f77ca539c/info/9213f4d7ee9b4fda857a97603a001f9e.8bbd7388262dc8cb1ce2cf4f04a7281d-
 hdfs://sv4r11s38:7000/hbase/TestTable/8bbd7388262dc8cb1ce2cf4f04a7281d/info/9213f4d7ee9b4fda857a97603a001f9e-bottom,
  keycount=416691, bloomtype=NONE, size=412.9m
 2011-11-15 05:48:00,690 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: 
 Found 3 hlogs to remove out of total 12; oldest outstanding sequenceid is 
 5699 from region 8bbd7388262dc8cb1ce2cf4f04a7281d
 2011-11-15 05:57:54,083 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: 
 Too many hlogs: logs=33, maxlogs=32; forcing flush of 1 regions(s): 
 8bbd7388262dc8cb1ce2cf4f04a7281d
 2011-11-15 05:57:54,083 WARN org.apache.hadoop.hbase.regionserver.LogRoller: 
 Failed to schedule flush of 8bbd7388262dc8cb1ce2cf4f04a7281dr=null, 
 requester=null
 2011-11-15 05:58:01,358 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: 
 Too many hlogs: logs=34, maxlogs=32; forcing flush of 1 regions(s): 
 8bbd7388262dc8cb1ce2cf4f04a7281d
 2011-11-15 05:58:01,359 WARN org.apache.hadoop.hbase.regionserver.LogRoller: 
 Failed to schedule flush of 8bbd7388262dc8cb1ce2cf4f04a7281dr=null, 
 requester=null
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4739) Master dying while going to close a region can leave it in transition forever

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159778#comment-13159778
 ] 

Hudson commented on HBASE-4739:
---

Integrated in HBase-0.92 #163 (See 
[https://builds.apache.org/job/HBase-0.92/163/])
HBASE-4739  Master dying while going to close a region can leave it in 
transition forever
   (Gao Jinchao)

tedyu : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/UnAssignCallable.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java


 Master dying while going to close a region can leave it in transition forever
 -

 Key: HBASE-4739
 URL: https://issues.apache.org/jira/browse/HBASE-4739
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Assignee: gaojinchao
Priority: Minor
 Fix For: 0.92.0, 0.94.0

 Attachments: 4739_trial2.patch, 4739_trialV3.patch, 
 HBASE-4739_Branch092.patch, HBASE-4739_Trunk.patch, 
 HBASE-4739_Trunk_V2.patch, HBASE-4739_V7.patch, HBASE-4739_trail5.patch, 
 HBASE-4739_trial.patch, HBASE-4739_trial6.patch


 I saw this in the aftermath of HBASE-4729 on a 0.92 refreshed yesterday, when 
 the master died it had just created the RIT znode for a region but didn't 
 tell the RS to close it yet.
 When the master restarted it saw the znode and started printing this:
 {quote}
 2011-11-03 00:02:49,130 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed 
 out:  TestTable,0007560564,1320253568406.f76899564cabe7e9857c3aeb526ec9dc. 
 state=CLOSING, ts=1320253605285, server=sv4r11s38,62003,1320195046948
 2011-11-03 00:02:49,130 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Region has been CLOSING for 
 too long, this should eventually complete or the server will expire, doing 
 nothing
 {quote}
 It's never going to happen, and it's blocking balancing.
 I'm marking this as minor since I believe this situation is pretty rare 
 unless you hit other bugs while trying out stuff to root bugs out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4773) HBaseAdmin may leak ZooKeeper connections

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159780#comment-13159780
 ] 

Hudson commented on HBASE-4773:
---

Integrated in HBase-0.92 #163 (See 
[https://builds.apache.org/job/HBase-0.92/163/])
HBASE-4773  HBaseAdmin may leak ZooKeeper connections (Xufeng)

tedyu : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java


 HBaseAdmin may leak ZooKeeper connections
 -

 Key: HBASE-4773
 URL: https://issues.apache.org/jira/browse/HBASE-4773
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.4
Reporter: gaojinchao
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.5

 Attachments: 4773.patch, branches_4773.patch, trunk_4773_patch.patch


 When master crashs, HBaseAdmin will leaks ZooKeeper connections
 I think we should close the zk connetion when throw MasterNotRunningException
  public HBaseAdmin(Configuration c)
   throws MasterNotRunningException, ZooKeeperConnectionException {
 this.conf = HBaseConfiguration.create(c);
 this.connection = HConnectionManager.getConnection(this.conf);
 this.pause = this.conf.getLong(hbase.client.pause, 1000);
 this.numRetries = this.conf.getInt(hbase.client.retries.number, 10);
 this.retryLongerMultiplier = 
 this.conf.getInt(hbase.client.retries.longer.multiplier, 10);
 //we should add this code and close the zk connection
 try{
   this.connection.getMaster();
 }catch(MasterNotRunningException e){
   HConnectionManager.deleteConnection(conf, false);
   throw e;  
 }
   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4785) Improve recovery time of HBase client when a region server dies.

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159779#comment-13159779
 ] 

Hudson commented on HBASE-4785:
---

Integrated in HBase-0.92 #163 (See 
[https://builds.apache.org/job/HBase-0.92/163/])
HBASE-4785 Improve recovery time of HBase client when a region server dies

stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java


 Improve recovery time of HBase client when a region server dies.
 

 Key: HBASE-4785
 URL: https://issues.apache.org/jira/browse/HBASE-4785
 Project: HBase
  Issue Type: Improvement
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Priority: Minor
 Fix For: 0.92.0

 Attachments: HBASE-4785.patch, HBASE-4785.patch


 When a region server dies, the HBase client waits until the RPC timesout 
 before learning that it needs to check META to find the new location of the 
 region. And it incurs this *timeout* cost for every region being served by 
 the dead region server. Remove this overhead by clearing the entries in cache 
 that have the dead region server as their values.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4877) TestHCM failing sporadically on jenkins and always for me on an ubuntu machine

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159782#comment-13159782
 ] 

Hudson commented on HBASE-4877:
---

Integrated in HBase-0.92 #163 (See 
[https://builds.apache.org/job/HBase-0.92/163/])
HBASE-4877 TestHCM failing sporadically on jenkins and always for me on an 
ubuntu machine

stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java


 TestHCM failing sporadically on jenkins and always for me on an ubuntu machine
 --

 Key: HBASE-4877
 URL: https://issues.apache.org/jira/browse/HBASE-4877
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.92.0

 Attachments: 4877.txt


 TestHCM takes 13 minutes for me on ubuntu and fails in testClosing.  It runs 
 fine on a mac.  The problem test is not testClosing as I thought originally, 
 its the test just previous, testConnectionUniqueness.  
 testConnectionUniqueness creates the maximum cached HConnections + 10 to 
 verify each is unique if the passed in Configuration has a unique hash.  
 Problem comes when zk enforces its default max from single host of 30 
 connections which is  (max cached + 10).  The max does not seem to be 
 enforced on mac for me.  The max connections runs up to max of 31 -- zk max + 
 1 -- and works fine until we do the +10.  On ubuntu, when we hit the zk max 
 of 30, we'll then go into a fail mode where we cannot set up a zk session... 
 each attempt takes a while.  Test passes, it just takes a while.
 Only, the uniqueness test does not clean up after itself and so all sessions 
 to zk are outstanding so then when the subsequent testClosing runs, it can't 
 set up connections successfully so fails.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4308) Race between RegionOpenedHandler and AssignmentManager

2011-11-29 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HBASE-4308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159787#comment-13159787
 ] 

Hudson commented on HBASE-4308:
---

Integrated in HBase-0.92 #163 (See 
[https://builds.apache.org/job/HBase-0.92/163/])
HBASE-4308 Race between RegionOpenedHandler and AssignmentManager (Ram)

ramkrishna : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java


 Race between RegionOpenedHandler and AssignmentManager
 --

 Key: HBASE-4308
 URL: https://issues.apache.org/jira/browse/HBASE-4308
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.92.0

 Attachments: HBASE-4308.patch, HBASE-4308_1.patch, HBASE-4308_2.patch


 When the master is processing a ZK event for REGION_OPENED, it calls delete() 
 on the znode before it removes the node from RegionsInTransition. If the 
 notification of that delete comes back into AssignmentManager before the 
 region is removed from RIT, you see an error like:
 2011-08-30 17:43:29,537 WARN  [main-EventThread] 
 master.AssignmentManager(861): Node deleted but still in RIT: 
 .META.,,1.1028785192 state=OPEN, ts=1314751409532, 
 server=todd-w510,55655,1314751396840
 Not certain if it causes issues, but it's a concerning log message.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

  1   2   >