date:20120325


 [ 
https://issues.apache.org/jira/browse/HBASE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-5190:
--

Attachment: 5190.addendum

Suggested addendum.

@J-D:
Please take a look.

 Limit the IPC queue size based on calls' payload size
 -

 Key: HBASE-5190
 URL: https://issues.apache.org/jira/browse/HBASE-5190
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
 Fix For: 0.94.0, 0.96.0

 Attachments: 5190.addendum, HBASE-5190-v2.patch, HBASE-5190-v3.patch, 
 HBASE-5190.patch


 Currently we limit the number of calls in the IPC queue only on their count. 
 It used to be really high and was dropped down recently to num_handlers * 10 
 (so 100 by default) because it was easy to OOME yourself when huge calls were 
 being queued. It's still possible to hit this problem if you use really big 
 values and/or a lot of handlers, so the idea is that we should take into 
 account the payload size. I can see 3 solutions:
  - Do the accounting outside of the queue itself for all calls coming in and 
 out and when a call doesn't fit, throw a retryable exception.
  - Same accounting but instead block the call when it comes in until space is 
 made available.
  - Add a new parameter for the maximum size (in bytes) of a Call and then set 
 the size the IPC queue (in terms of the number of items) so that it could 
 only contain as many items as some predefined maximum size (in bytes) for the 
 whole queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5633) NPE reading ZK config in HBase

2012-03-25 Thread Matteo Bertozzi (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5633:
---

Status: Patch Available  (was: Reopened)

 NPE reading ZK config in HBase
 --

 Key: HBASE-5633
 URL: https://issues.apache.org/jira/browse/HBASE-5633
 Project: HBase
  Issue Type: Bug
  Components: zookeeper
Reporter: Matteo Bertozzi
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, 
 HBASE-5633-v1.patch, HBASE-5633-v2.patch


 If zoo.cfg contains server.* (server.0=server0:2888:3888\n) and 
 cluster.distributed property (in hbase-site.xml) is empty we get an NPE in 
 parseZooCfg().
 The easy way to reproduce the bug is running 
 org.apache.hbase.zookeeper.TestHQuorumPeer with hbase-site.xml containing:
 {code}
 property
   namehbase.cluster.distributed/name
   value/value
 /property
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HBASE-5633) NPE reading ZK config in HBase

2012-03-25 Thread Matteo Bertozzi (Reopened) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi reopened HBASE-5633:


  Assignee: (was: Matteo Bertozzi)

 NPE reading ZK config in HBase
 --

 Key: HBASE-5633
 URL: https://issues.apache.org/jira/browse/HBASE-5633
 Project: HBase
  Issue Type: Bug
  Components: zookeeper
Reporter: Matteo Bertozzi
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, 
 HBASE-5633-v1.patch, HBASE-5633-v2.patch


 If zoo.cfg contains server.* (server.0=server0:2888:3888\n) and 
 cluster.distributed property (in hbase-site.xml) is empty we get an NPE in 
 parseZooCfg().
 The easy way to reproduce the bug is running 
 org.apache.hbase.zookeeper.TestHQuorumPeer with hbase-site.xml containing:
 {code}
 property
   namehbase.cluster.distributed/name
   value/value
 /property
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5633) NPE reading ZK config in HBase

[
https://issues.apache.org/jira/browse/HBASE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237957#comment-13237957
]

Hadoop QA commented on HBASE-5633:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519868/HBASE-5633-0.92.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1300//console

This message is automatically generated.

NPE reading ZK config in HBase
--

Key: HBASE-5633
URL: https://issues.apache.org/jira/browse/HBASE-5633
Project: HBase
Issue Type: Bug
Components: zookeeper
Reporter: Matteo Bertozzi
Priority: Minor
Fix For: 0.94.0

Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch,
HBASE-5633-v1.patch, HBASE-5633-v2.patch

If zoo.cfg contains server.* (server.0=server0:2888:3888\n) and
cluster.distributed property (in hbase-site.xml) is empty we get an NPE in
parseZooCfg().
The easy way to reproduce the bug is running
org.apache.hbase.zookeeper.TestHQuorumPeer with hbase-site.xml containing:
{code}
property
namehbase.cluster.distributed/name
value/value
/property
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5217) Reenable the thrift tests, and add a new one for getRegionInfo


 [ 
https://issues.apache.org/jira/browse/HBASE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-5217:
---

Status: In Progress  (was: Patch Available)

 Reenable the thrift tests, and add a new one for getRegionInfo
 --

 Key: HBASE-5217
 URL: https://issues.apache.org/jira/browse/HBASE-5217
 Project: HBase
  Issue Type: Improvement
Reporter: Alex Newman
Assignee: Alex Newman
Priority: Minor
 Attachments: 0001-Fixing-thrift-tests-v2.patch, 
 0001-Fixing-thrift-tests.patch, 
 0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-.patch, 
 -hbase-posix4e #92 Console [Jenkins].pdf


 At some point we disabled tests for the thrift server. In addition, it looks 
 like the getRegionInfo no longer functions. I'd like to reenable the tests 
 and add one for getRegionInfo. I had to write this to test my changes in 
 HBASE-2600 anyway. I figured I would break it out. We shouldn't commit it 
 until we have fixed getting the regioninfo from the thriftserver.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5217) Reenable the thrift tests, and add a new one for getRegionInfo


 [ 
https://issues.apache.org/jira/browse/HBASE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-5217:
---

Attachment: (was: 
0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-v2.patch)

 Reenable the thrift tests, and add a new one for getRegionInfo
 --

 Key: HBASE-5217
 URL: https://issues.apache.org/jira/browse/HBASE-5217
 Project: HBase
  Issue Type: Improvement
Reporter: Alex Newman
Assignee: Alex Newman
Priority: Minor
 Attachments: 0001-Fixing-thrift-tests-v2.patch, 
 0001-Fixing-thrift-tests.patch, 
 0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-.patch, 
 -hbase-posix4e #92 Console [Jenkins].pdf


 At some point we disabled tests for the thrift server. In addition, it looks 
 like the getRegionInfo no longer functions. I'd like to reenable the tests 
 and add one for getRegionInfo. I had to write this to test my changes in 
 HBASE-2600 anyway. I figured I would break it out. We shouldn't commit it 
 until we have fixed getting the regioninfo from the thriftserver.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4382) Region encoded name is hash of tablename + start key + regionid (timestamp); should include end key when hashing.


 [ 
https://issues.apache.org/jira/browse/HBASE-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4382:
---

Attachment: (was: 
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch)

 Region encoded name is hash of tablename + start key + regionid (timestamp); 
 should include end key when hashing.
 -

 Key: HBASE-4382
 URL: https://issues.apache.org/jira/browse/HBASE-4382
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: Alex Newman
  Labels: noob
 Attachments: 
 0003-Verify-start-and-end-key-are-contained-in-the-encode.patch


 Seems odd that region encoded name is same for regions if made in same second 
 with same start key tough their end keys are different.  It can happen in 
 unit test.  Should mix in the end key when coming up w/ the region name 
 encoded name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5217) Reenable the thrift tests, and add a new one for getRegionInfo


 [ 
https://issues.apache.org/jira/browse/HBASE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-5217:
---

Attachment: 0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-.patch

v2

 Reenable the thrift tests, and add a new one for getRegionInfo
 --

 Key: HBASE-5217
 URL: https://issues.apache.org/jira/browse/HBASE-5217
 Project: HBase
  Issue Type: Improvement
Reporter: Alex Newman
Assignee: Alex Newman
Priority: Minor
 Attachments: 0001-Fixing-thrift-tests-v2.patch, 
 0001-Fixing-thrift-tests.patch, 
 0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-.patch, 
 -hbase-posix4e #92 Console [Jenkins].pdf


 At some point we disabled tests for the thrift server. In addition, it looks 
 like the getRegionInfo no longer functions. I'd like to reenable the tests 
 and add one for getRegionInfo. I had to write this to test my changes in 
 HBASE-2600 anyway. I figured I would break it out. We shouldn't commit it 
 until we have fixed getting the regioninfo from the thriftserver.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4382) Region encoded name is hash of tablename + start key + regionid (timestamp); should include end key when hashing.

2012-03-25 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-4382:
---

Attachment: 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch

 Region encoded name is hash of tablename + start key + regionid (timestamp); 
 should include end key when hashing.
 -

 Key: HBASE-4382
 URL: https://issues.apache.org/jira/browse/HBASE-4382
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: Alex Newman
  Labels: noob
 Attachments: 
 0003-Verify-start-and-end-key-are-contained-in-the-encode.patch


 Seems odd that region encoded name is same for regions if made in same second 
 with same start key tough their end keys are different.  It can happen in 
 unit test.  Should mix in the end key when coming up w/ the region name 
 encoded name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5217) Reenable the thrift tests, and add a new one for getRegionInfo

[
https://issues.apache.org/jira/browse/HBASE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237964#comment-13237964
]

jirapos...@reviews.apache.org commented on HBASE-5217:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3514/
---

(Updated 2012-03-25 20:11:30.928544)

Review request for hbase.

Summary
---

At some point we disabled tests for the thrift server. In addition, it looks
like the getRegionInfo no longer functions. I'd like to reenable the tests and
add one for getRegionInfo.

This addresses bug HBASE-5217.
https://issues.apache.org/jira/browse/HBASE-5217

Diffs (updated)
-

src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java 60eb426
src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java 444d6d5

Diff: https://reviews.apache.org/r/3514/diff

Testing
---

Ran the tests with my changes in HBASE-2600 to test.

Thanks,

Alex

Reenable the thrift tests, and add a new one for getRegionInfo
--

Key: HBASE-5217
URL: https://issues.apache.org/jira/browse/HBASE-5217
Project: HBase
Issue Type: Improvement
Reporter: Alex Newman
Assignee: Alex Newman
Priority: Minor
Attachments: 0001-Fixing-thrift-tests-v2.patch,
0001-Fixing-thrift-tests.patch,
0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-.patch,
-hbase-posix4e #92 Console [Jenkins].pdf

At some point we disabled tests for the thrift server. In addition, it looks
like the getRegionInfo no longer functions. I'd like to reenable the tests
and add one for getRegionInfo. I had to write this to test my changes in
HBASE-2600 anyway. I figured I would break it out. We shouldn't commit it
until we have fixed getting the regioninfo from the thriftserver.

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

2012-03-25 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-2600:
---

Attachment: 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237965#comment-13237965
 ] 

jirapos...@reviews.apache.org commented on HBASE-2600:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3466/
---

(Updated 2012-03-25 20:11:32.746962)


Review request for hbase, Michael Stack and Lars Hofhansl.


Summary (updated)
---

This is an idea that Ryan and I have been kicking around on and off for a while 
now.

If regionnames were made of tablename+endrow instead of tablename+startrow, 
then in the metatables, doing a search for the region that contains the wanted 
row, we'd just have to open a scanner using passed row and the first row found 
by the scan would be that of the region we need (If offlined parent, we'd have 
to scan to the next row).

If we redid the meta tables in this format, we'd be using an access that is 
natural to hbase, a scan as opposed to the perverse, expensive 
getClosestRowBefore we currently have that has to walk backward in meta finding 
a containing region.

This issue is about changing the way we name regions.

If we were using scans, prewarming client cache would be near costless (as 
opposed to what we'll currently have to do which is first a getClosestRowBefore 
and then a scan from the closestrowbefore forward).

Converting to the new method, we'd have to run a migration on startup changing 
the content in meta.

Up to this, the randomid component of a region name has been the timestamp of 
region creation. HBASE-2531 32-bit encoding of regionnames waaay too 
susceptible to hash clashes proposes changing the randomid so that it contains 
actual name of the directory in the filesystem that hosts the region. If we had 
this in place, I think it would help with the migration to this new way of 
doing the meta because as is, the region name in fs is a hash of regionname... 
changing the format of the regionname would mean we generate a different 
hash... so we'd need hbase-2531 to be in place before we could do this change.


public TRegionInfo getRegionInfo(ByteBuffer searchRow) throws IOError { was 
nulled out and enabled with https://reviews.apache.org/r/3514/. They are listed 
as dependencies in the jira and will be committed together.


This addresses bug HBASE-2600.
https://issues.apache.org/jira/browse/HBASE-2600


Diffs (updated)
-

  
security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
 c1f20de 
  src/main/java/org/apache/hadoop/hbase/HConstants.java 347 
  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 8d83ff3 
  src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java fc5e53e 
  src/main/java/org/apache/hadoop/hbase/KeyValue.java 243d76f 
  src/main/java/org/apache/hadoop/hbase/catalog/MetaMigratev2.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
  src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
  src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java b2a5463 
  src/main/java/org/apache/hadoop/hbase/client/HTable.java 8e7d7f7 
  src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java 04150ad 
  src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 47381f4 
  src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java f404999 
  src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 
197eb71 
  src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java 
18c13c4 
  src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java 30c61ca 
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 757f98e 
  src/main/java/org/apache/hadoop/hbase/master/HMaster.java dbc9251 
  
src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 
2ec6677 
  src/main/java/org/apache/hadoop/hbase/migration/HRegionInfo090x2.java 
PRE-CREATION 
  
src/main/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java
 8174cf5 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 02d55d4 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java e0af8fb 
  src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java 0592f40 
  src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java 
a3850e5 
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 0c7b396 
  src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java 56e31e1 
  src/main/java/org/apache/hadoop/hbase/rest/model/TableRegionModel.java 
3535595 
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java 60eb426 
  src/main/java/org/apache/hadoop/hbase/thrift/generated/AlreadyExists.java 
a5b81f5

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

[
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237968#comment-13237968
]

Hadoop QA commented on HBASE-2600:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12519877/0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 64 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1301//console

This message is automatically generated.

Change how we do meta tables; from tablename+STARTROW+randomid to instead,
tablename+ENDROW+randomid

Key: HBASE-2600
URL: https://issues.apache.org/jira/browse/HBASE-2600
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
Attachments:
0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
2600-trunk-01-17.txt, jenkins.pdf

This is an idea that Ryan and I have been kicking around on and off for a
while now.
If regionnames were made of tablename+endrow instead of tablename+startrow,
then in the metatables, doing a search for the region that contains the
wanted row, we'd just have to open a scanner using passed row and the first
row found by the scan would be that of the region we need (If offlined
parent, we'd have to scan to the next row).
If we redid the meta tables in this format, we'd be using an access that is
natural to hbase, a scan as opposed to the perverse, expensive
getClosestRowBefore we currently have that has to walk backward in meta
finding a containing region.
This issue is about changing the way we name regions.
If we were using scans, prewarming client cache would be near costless (as
opposed to what we'll currently have to do which is first a
getClosestRowBefore and then a scan from the closestrowbefore forward).
Converting to the new method, we'd have to run a migration on startup
changing the content in meta.
Up to this, the randomid component of a region name has been the timestamp of
region creation. HBASE-2531 32-bit encoding of regionnames waaay
too susceptible to hash clashes proposes changing the randomid so that it
contains actual name of the directory in the filesystem that hosts the
region. If we had this in place, I think it would help with the migration to
this new way of doing the meta because as is, the region name in fs is a hash
of regionname... changing the format of the regionname would mean we generate
a different hash... so we'd need hbase-2531 to be in place before we could do
this change.

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


 [ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-2600:
---

Attachment: HBASE-2600+5217-Sun-Mar-25-2012.patch

Combined with 5217 as they need to be committed at the same time.

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012.patch, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

[
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237973#comment-13237973
]

Hadoop QA commented on HBASE-2600:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12519878/HBASE-2600%2B5217-Sun-Mar-25-2012.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1302//console

This message is automatically generated.

Change how we do meta tables; from tablename+STARTROW+randomid to instead,
tablename+ENDROW+randomid

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


 [ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-2600:
---

Attachment: HBASE-2600+5217-Sun-Mar-25-2012-v2.patch

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v2.patch, 
 HBASE-2600+5217-Sun-Mar-25-2012.patch, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


 [ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-2600:
---

Attachment: (was: HBASE-2600+5217-Sun-Mar-25-2012.patch)

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

2012-03-25 Thread Alex Newman (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237974#comment-13237974
 ] 

Alex Newman commented on HBASE-2600:


Sorry I apparently don't know how to generate patches with git. That was just 
the HBASE-2600 patch.

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


 [ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-2600:
---

Attachment: (was: HBASE-2600+5217-Sun-Mar-25-2012-v2.patch)

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


 [ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-2600:
---

Attachment: HBASE-2600+5217-Sun-Mar-25-2012-v3.patch

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5190) Limit the IPC queue size based on calls' payload size

2012-03-25 Thread Lars Hofhansl (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237977#comment-13237977
 ] 

Lars Hofhansl commented on HBASE-5190:
--

+1 on addendum

 Limit the IPC queue size based on calls' payload size
 -

 Key: HBASE-5190
 URL: https://issues.apache.org/jira/browse/HBASE-5190
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
 Fix For: 0.94.0, 0.96.0

 Attachments: 5190.addendum, HBASE-5190-v2.patch, HBASE-5190-v3.patch, 
 HBASE-5190.patch


 Currently we limit the number of calls in the IPC queue only on their count. 
 It used to be really high and was dropped down recently to num_handlers * 10 
 (so 100 by default) because it was easy to OOME yourself when huge calls were 
 being queued. It's still possible to hit this problem if you use really big 
 values and/or a lot of handlers, so the idea is that we should take into 
 account the payload size. I can see 3 solutions:
  - Do the accounting outside of the queue itself for all calls coming in and 
 out and when a call doesn't fit, throw a retryable exception.
  - Same accounting but instead block the call when it comes in until space is 
 made available.
  - Add a new parameter for the maximum size (in bytes) of a Call and then set 
 the size the IPC queue (in terms of the number of items) so that it could 
 only contain as many items as some predefined maximum size (in bytes) for the 
 whole queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

[
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237998#comment-13237998
]

Hadoop QA commented on HBASE-2600:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12519880/HBASE-2600%2B5217-Sun-Mar-25-2012-v3.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 50 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 11 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
org.apache.hadoop.hbase.catalog.TestMetaUpdate
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1303//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1303//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1303//console

This message is automatically generated.

Change how we do meta tables; from tablename+STARTROW+randomid to instead,
tablename+ENDROW+randomid

Key: HBASE-2600
URL: https://issues.apache.org/jira/browse/HBASE-2600
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
Attachments:
0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, jenkins.pdf

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

2012-03-25 Thread Alex Newman (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238004#comment-13238004
 ] 

Alex Newman commented on HBASE-2600:


I bungled the patch. I've been using git diff --no-prefix HEAD^^  bla . That 
doesn't seem to include my binary tar ball. ANy ideas

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5615) the master never does balance because of balancing the parent region


 [ 
https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-5615:
--

Fix Version/s: 0.94.0
   0.92.2

 the master never does balance because of balancing the parent region
 

 Key: HBASE-5615
 URL: https://issues.apache.org/jira/browse/HBASE-5615
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.7
Reporter: xufeng
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, 
 NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html


 the master never do balance becauseof when master do rebuildUserRegions()，it 
 will add the parent region into  AssignmentManager#servers,
 if balancer let the parent region to move,the parent will in RIT forever.thus 
 balance will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region


[ 
https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238009#comment-13238009
 ] 

Ted Yu commented on HBASE-5615:
---

Integrated to 0.92, 0.94 and TRUNK as well.

 the master never does balance because of balancing the parent region
 

 Key: HBASE-5615
 URL: https://issues.apache.org/jira/browse/HBASE-5615
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.7
Reporter: xufeng
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, 
 NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html


 the master never do balance becauseof when master do rebuildUserRegions()，it 
 will add the parent region into  AssignmentManager#servers,
 if balancer let the parent region to move,the parent will in RIT forever.thus 
 balance will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238010#comment-13238010
 ] 

Ted Yu commented on HBASE-2600:
---

@Alex:
Try this:
{code}
git diff --no-prefix --binary 
{code}

Thanks

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

[
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ted Yu updated HBASE-2600:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12510928/0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 31 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/800//console

This message is automatically generated.)

Change how we do meta tables; from tablename+STARTROW+randomid to instead,
tablename+ENDROW+randomid

Key: HBASE-2600
URL: https://issues.apache.org/jira/browse/HBASE-2600
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
Attachments:
0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, jenkins.pdf

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

[
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ted Yu updated HBASE-2600:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12519877/0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 64 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1301//console

This message is automatically generated.)

Change how we do meta tables; from tablename+STARTROW+randomid to instead,
tablename+ENDROW+randomid

Key: HBASE-2600
URL: https://issues.apache.org/jira/browse/HBASE-2600
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
Attachments:
0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, jenkins.pdf

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

[
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ted Yu updated HBASE-2600:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12519878/HBASE-2600%2B5217-Sun-Mar-25-2012.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1302//console

This message is automatically generated.)

Change how we do meta tables; from tablename+STARTROW+randomid to instead,
tablename+ENDROW+randomid

Key: HBASE-2600
URL: https://issues.apache.org/jira/browse/HBASE-2600
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
Attachments:
0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, jenkins.pdf

[jira] [Commented] (HBASE-5190) Limit the IPC queue size based on calls' payload size

2012-03-25 Thread stack (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238014#comment-13238014
 ] 

stack commented on HBASE-5190:
--

Addendum looks good to me. +1

 Limit the IPC queue size based on calls' payload size
 -

 Key: HBASE-5190
 URL: https://issues.apache.org/jira/browse/HBASE-5190
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
 Fix For: 0.94.0, 0.96.0

 Attachments: 5190.addendum, HBASE-5190-v2.patch, HBASE-5190-v3.patch, 
 HBASE-5190.patch


 Currently we limit the number of calls in the IPC queue only on their count. 
 It used to be really high and was dropped down recently to num_handlers * 10 
 (so 100 by default) because it was easy to OOME yourself when huge calls were 
 being queued. It's still possible to hit this problem if you use really big 
 values and/or a lot of handlers, so the idea is that we should take into 
 account the payload size. I can see 3 solutions:
  - Do the accounting outside of the queue itself for all calls coming in and 
 out and when a call doesn't fit, throw a retryable exception.
  - Same accounting but instead block the call when it comes in until space is 
 made available.
  - Add a new parameter for the maximum size (in bytes) of a Call and then set 
 the size the IPC queue (in terms of the number of items) so that it could 
 only contain as many items as some predefined maximum size (in bytes) for the 
 whole queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region

2012-03-25 Thread xufeng (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238029#comment-13238029
 ] 

xufeng commented on HBASE-5615:
---

Thanks for help Ramkrishna,Jinchao and Ted.

 the master never does balance because of balancing the parent region
 

 Key: HBASE-5615
 URL: https://issues.apache.org/jira/browse/HBASE-5615
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.7
Reporter: xufeng
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, 
 NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html


 the master never do balance becauseof when master do rebuildUserRegions()，it 
 will add the parent region into  AssignmentManager#servers,
 if balancer let the parent region to move,the parent will in RIT forever.thus 
 balance will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region


[ 
https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238030#comment-13238030
 ] 

Ted Yu commented on HBASE-5615:
---

@Xufeng:
You're welcome.

In the future, please grant license to Apache when you attach patches.

 the master never does balance because of balancing the parent region
 

 Key: HBASE-5615
 URL: https://issues.apache.org/jira/browse/HBASE-5615
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.7
Reporter: xufeng
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, 
 NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html


 the master never do balance becauseof when master do rebuildUserRegions()，it 
 will add the parent region into  AssignmentManager#servers,
 if balancer let the parent region to move,the parent will in RIT forever.thus 
 balance will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4147) StoreFile query usage report

2012-03-25 Thread Otis Gospodnetic (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238035#comment-13238035
]

Otis Gospodnetic commented on HBASE-4147:
-

Yes, +1 for publishing in JMX. We need that for our SPM for HBase monitoring
service, for example.

StoreFile query usage report

Key: HBASE-4147
URL: https://issues.apache.org/jira/browse/HBASE-4147
Project: HBase
Issue Type: Improvement
Reporter: Doug Meil
Priority: Critical
Fix For: 0.96.0

Attachments: hbase_4147_storefilereport.pdf,
hbase_4147_storefilereport_2011_08_10.pdf

Detailed information on what HBase is doing in terms of reads is hard to come
by.
What would be useful is to have a periodic StoreFile query report.
Specifically, this could run on a configured interval (e.g., every 30
seconds, 60 seconds) and dump the output to the log files.
This would have all StoreFiles accessed during the reporting period (and with
the Path we would also know region, CF, and table), # of times the StoreFile
was accessed, the size of the StoreFile, and the total time (ms) spent
processing that StoreFile.
Even this level of summary would be useful to detect a which tables CFs are
being accessed the most, and including the StoreFile would provide insight
into relative uncompaction (i.e., lots of StoreFiles).
I think the log-output, as opposed to UI, is an important facet with this.
I'm assuming that users will slice and dice this data on their own so I think
we should skip any kind of admin view for now (i.e., new JSPs, new APIs to
expose this data). Just getting this to log-file would be a big improvement.
Will this have a non-zero performance impact? Yes. Hopefully small, but yes
it will. However, flying a plane without any instrumentation isn't fun. :-)

[jira] [Updated] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


 [ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Newman updated HBASE-2600:
---

Attachment: HBASE-2600+5217-Sun-Mar-25-2012-v4.patch

Binary included

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, 
 HBASE-2600+5217-Sun-Mar-25-2012-v4.patch, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4957) Clean up some log messages, code in RecoverableZooKeeper


[ 
https://issues.apache.org/jira/browse/HBASE-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238040#comment-13238040
 ] 

Hudson commented on HBASE-4957:
---

Integrated in HBase-TRUNK #2695 (See 
[https://builds.apache.org/job/HBase-TRUNK/2695/])
HBASE-4957 Clean up some log messages, code in RecoverableZooKeeper (Todd) 
(Revision 1304940)

 Result = FAILURE
larsh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/RetryCounter.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java


 Clean up some log messages, code in RecoverableZooKeeper
 

 Key: HBASE-4957
 URL: https://issues.apache.org/jira/browse/HBASE-4957
 Project: HBase
  Issue Type: Improvement
  Components: zookeeper
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: hbase-4957.txt, hbase-4957.txt, hbase-4957.txt, 
 hbase-4957.txt


 In RecoverableZooKeeper, there are a number of log messages and comments 
 which don't really read correctly, and some other pieces of code that can be 
 cleaned up. Simple cleanup - shouldn't be any actual behavioral changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5633) NPE reading ZK config in HBase


[ 
https://issues.apache.org/jira/browse/HBASE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238041#comment-13238041
 ] 

Hudson commented on HBASE-5633:
---

Integrated in HBase-TRUNK #2695 (See 
[https://builds.apache.org/job/HBase-TRUNK/2695/])
HBASE-5633 NPE reading ZK config in HBase (Revision 1304924)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java


 NPE reading ZK config in HBase
 --

 Key: HBASE-5633
 URL: https://issues.apache.org/jira/browse/HBASE-5633
 Project: HBase
  Issue Type: Bug
  Components: zookeeper
Reporter: Matteo Bertozzi
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, 
 HBASE-5633-v1.patch, HBASE-5633-v2.patch


 If zoo.cfg contains server.* (server.0=server0:2888:3888\n) and 
 cluster.distributed property (in hbase-site.xml) is empty we get an NPE in 
 parseZooCfg().
 The easy way to reproduce the bug is running 
 org.apache.hbase.zookeeper.TestHQuorumPeer with hbase-site.xml containing:
 {code}
 property
   namehbase.cluster.distributed/name
   value/value
 /property
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region


[ 
https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238042#comment-13238042
 ] 

Hudson commented on HBASE-5615:
---

Integrated in HBase-TRUNK #2695 (See 
[https://builds.apache.org/job/HBase-TRUNK/2695/])
HBASE-5615 the master never does balance because of balancing the parent 
region (Xufeng) (Revision 1305171)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java


 the master never does balance because of balancing the parent region
 

 Key: HBASE-5615
 URL: https://issues.apache.org/jira/browse/HBASE-5615
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.7
Reporter: xufeng
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, 
 NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html


 the master never do balance becauseof when master do rebuildUserRegions()，it 
 will add the parent region into  AssignmentManager#servers,
 if balancer let the parent region to move,the parent will in RIT forever.thus 
 balance will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5434) [REST] Include more metrics in cluster status request


[ 
https://issues.apache.org/jira/browse/HBASE-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238043#comment-13238043
 ] 

Hudson commented on HBASE-5434:
---

Integrated in HBase-TRUNK #2695 (See 
[https://builds.apache.org/job/HBase-TRUNK/2695/])
HBASE-5434 [REST] Include more metrics in cluster status request (Mubarak 
Seyed) (Revision 1304919)

 Result = FAILURE
larsh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HServerLoad.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/model/StorageClusterStatusModel.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/StorageClusterStatusMessage.java
* /hbase/trunk/src/main/resources/org/apache/hadoop/hbase/rest/XMLSchema.xsd
* 
/hbase/trunk/src/main/resources/org/apache/hadoop/hbase/rest/protobuf/StorageClusterStatusMessage.proto
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/rest/model/TestStorageClusterStatusModel.java


 [REST] Include more metrics in cluster status request
 -

 Key: HBASE-5434
 URL: https://issues.apache.org/jira/browse/HBASE-5434
 Project: HBase
  Issue Type: Improvement
  Components: metrics, rest
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Assignee: Mubarak Seyed
Priority: Minor
  Labels: noob
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5434.trunk.v1.patch, HBASE-5434.trunk.v2.patch, 
 HBASE-5434.trunk.v2.patch, HBASE-5434.trunk.v2.patch


 /status/cluster shows only
 {code}
 stores=2
 storefiless=0
 storefileSizeMB=0
 memstoreSizeMB=0
 storefileIndexSizeMB=0
 {code}
 for a region but master web-ui shows
 {code}
 stores=1,
 storefiles=0,
 storefileUncompressedSizeMB=0
 storefileSizeMB=0
 memstoreSizeMB=0
 storefileIndexSizeMB=0
 readRequestsCount=0
 writeRequestsCount=0
 rootIndexSizeKB=0
 totalStaticIndexSizeKB=0
 totalStaticBloomSizeKB=0
 totalCompactingKVs=0
 currentCompactedKVs=0
 compactionProgressPct=NaN
 {code}
 In a write-heavy REST gateway based production environment, ops team needs to 
 verify whether write counters are getting incremented per region (they do run 
 /status/cluster on each REST server), we can get the same values from 
 *rpc.metrics.put_num_ops* and *hbase.regionserver.writeRequestsCount* but 
 some home-grown tools needs to parse the output of /status/cluster and 
 updates the dashboard.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region


[ 
https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238048#comment-13238048
 ] 

Hudson commented on HBASE-5615:
---

Integrated in HBase-0.94 #55 (See 
[https://builds.apache.org/job/HBase-0.94/55/])
HBASE-5615 the master never does balance because of balancing the parent 
region (Xufeng) (Revision 1305172)

 Result = SUCCESS
tedyu : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java


 the master never does balance because of balancing the parent region
 

 Key: HBASE-5615
 URL: https://issues.apache.org/jira/browse/HBASE-5615
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.7
Reporter: xufeng
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, 
 NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html


 the master never do balance becauseof when master do rebuildUserRegions()，it 
 will add the parent region into  AssignmentManager#servers,
 if balancer let the parent region to move,the parent will in RIT forever.thus 
 balance will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid

[
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238063#comment-13238063
]

Hadoop QA commented on HBASE-2600:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12519888/HBASE-2600%2B5217-Sun-Mar-25-2012-v4.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 48 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 11 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.catalog.TestMetaUpdate
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1304//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1304//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1304//console

This message is automatically generated.

Change how we do meta tables; from tablename+STARTROW+randomid to instead,
tablename+ENDROW+randomid

Key: HBASE-2600
URL: https://issues.apache.org/jira/browse/HBASE-2600
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
Attachments:
0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch,
0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch,
2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch,
HBASE-2600+5217-Sun-Mar-25-2012-v4.patch, jenkins.pdf

[jira] [Commented] (HBASE-5599) The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_

2012-03-25 Thread fulin wang (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238064#comment-13238064
]

fulin wang commented on HBASE-5599:
---

Today I will make a patch base on 0.90 new codes.
If I change in 0.90 that it pass the review, this code will merge into
0.92/0.94/trunk versions.

The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE,
NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED,
FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN.

Key: HBASE-5599
URL: https://issues.apache.org/jira/browse/HBASE-5599
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Fix For: 0.90.6

Attachments: hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch,
hbase-5599-0.90_v3.patch

The hbck tool can not fix the six scenarios.
1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any
region server.
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

4. [REGIONNAME] should not be deployed according to META, but is deployed on
[SERVERNAME]
Fix: Close this region.

5. First region should start with an empty key. You need to create a new
region and regioninfo in HDFS to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty
region for this error.
6. There is a hole in the region chain between [KEY] and [KEY]. You need to
create a new regioninfo and region dir in hdfs to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty region
for this hole.

[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid


[ 
https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238074#comment-13238074
 ] 

Ted Yu commented on HBASE-2600:
---

dev-support/test-patch.sh doesn't use '--binary' option when applying patches.

I tried the following command:
{code}
patch -p0 --binary -i HBASE-2600+5217-Sun-Mar-25-2012-v4.patch
{code}
But src/test/data/hbase-2600-root.dir.tgz wasn't unpacked from patch.

 Change how we do meta tables; from tablename+STARTROW+randomid to instead, 
 tablename+ENDROW+randomid
 

 Key: HBASE-2600
 URL: https://issues.apache.org/jira/browse/HBASE-2600
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Alex Newman
 Attachments: 
 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 
 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 
 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, 
 HBASE-2600+5217-Sun-Mar-25-2012-v4.patch, jenkins.pdf


 This is an idea that Ryan and I have been kicking around on and off for a 
 while now.
 If regionnames were made of tablename+endrow instead of tablename+startrow, 
 then in the metatables, doing a search for the region that contains the 
 wanted row, we'd just have to open a scanner using passed row and the first 
 row found by the scan would be that of the region we need (If offlined 
 parent, we'd have to scan to the next row).
 If we redid the meta tables in this format, we'd be using an access that is 
 natural to hbase, a scan as opposed to the perverse, expensive 
 getClosestRowBefore we currently have that has to walk backward in meta 
 finding a containing region.
 This issue is about changing the way we name regions.
 If we were using scans, prewarming client cache would be near costless (as 
 opposed to what we'll currently have to do which is first a 
 getClosestRowBefore and then a scan from the closestrowbefore forward).
 Converting to the new method, we'd have to run a migration on startup 
 changing the content in meta.
 Up to this, the randomid component of a region name has been the timestamp of 
 region creation.   HBASE-2531 32-bit encoding of regionnames waaay 
 too susceptible to hash clashes proposes changing the randomid so that it 
 contains actual name of the directory in the filesystem that hosts the 
 region.  If we had this in place, I think it would help with the migration to 
 this new way of doing the meta because as is, the region name in fs is a hash 
 of regionname... changing the format of the regionname would mean we generate 
 a different hash... so we'd need hbase-2531 to be in place before we could do 
 this change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region


[ 
https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238083#comment-13238083
 ] 

Hudson commented on HBASE-5615:
---

Integrated in HBase-0.92 #339 (See 
[https://builds.apache.org/job/HBase-0.92/339/])
HBASE-5615 the master never does balance because of balancing the parent 
region (Xufeng) (Revision 1305173)

 Result = FAILURE
tedyu : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java


 the master never does balance because of balancing the parent region
 

 Key: HBASE-5615
 URL: https://issues.apache.org/jira/browse/HBASE-5615
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.7
Reporter: xufeng
Assignee: xufeng
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, 
 NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html


 the master never do balance becauseof when master do rebuildUserRegions()，it 
 will add the parent region into  AssignmentManager#servers,
 if balancer let the parent region to move,the parent will in RIT forever.thus 
 balance will never be executed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5564) Bulkload is discarding duplicate records

2012-03-25 Thread Laxman (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13238095#comment-13238095
]

Laxman commented on HBASE-5564:
---

@Anoop, thanks for clarification.

@Stack, thanks for the review. I will update the patch.

bq. need curlies
bq. NO_TIMESTAMP_KEYCOLUMN_INDEX

I will update the patch for above 2 comments.

bq. Can you confirm that current behavior – setting ts to
System.currentTimeMillis – is default? It seems to be ... we set
System.currentTimeMillis as time to use setting up the job.

Before patch, we are setting ts to System.currentTimeMillis in
TsvImporterMapper.doSetup. This setup methos will be called for each mapper,
i.e, for each input split. That means it uses a new timestamp for each map task.

After patch, we are setting ts to conf.getLong which is same in all map tasks.

Hope, I understood your question correctly.

Bulkload is discarding duplicate records

Key: HBASE-5564
URL: https://issues.apache.org/jira/browse/HBASE-5564
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Environment: HBase 0.92
Reporter: Laxman
Assignee: Laxman
Labels: bulkloader
Fix For: 0.96.0

Attachments: 5564.lint, HBASE-5564_trunk.1.patch,
HBASE-5564_trunk.1.patch, HBASE-5564_trunk.patch

Duplicate records are getting discarded when duplicate records exists in same
input file and more specifically if they exists in same split.
Duplicate records are considered if the records are from diffrent different
splits.
Version under test: HBase 0.92

[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region