date:20120229


 [ 
https://issues.apache.org/jira/browse/HBASE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5490:
--

Attachment: 5490-v2.txt

I changed ordinal for RS_ZK_REGION_FAILED_OPEN to be larger than that of 
M_META_SERVER_SHUTDOWN.

 Move the enum RS_ZK_REGION_FAILED_OPEN to the last of the enum list in 0.90 
 EventHandler
 

 Key: HBASE-5490
 URL: https://issues.apache.org/jira/browse/HBASE-5490
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
 Fix For: 0.90.6

 Attachments: 5490-v2.txt, HBASE-5490.patch


 The new state that was added  RS_ZK_REGION_FAILED_OPEN was failing the 
 rolling restart.
 So move the new enum to the end of the list.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5490) Move the enum RS_ZK_REGION_FAILED_OPEN to the last of the enum list in 0.90 EventHandler


[ 
https://issues.apache.org/jira/browse/HBASE-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219240#comment-13219240
 ] 

Zhihong Yu commented on HBASE-5490:
---

Integrated patch v2 to 0.90

Thanks for the patch, Ram.

 Move the enum RS_ZK_REGION_FAILED_OPEN to the last of the enum list in 0.90 
 EventHandler
 

 Key: HBASE-5490
 URL: https://issues.apache.org/jira/browse/HBASE-5490
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
 Fix For: 0.90.6

 Attachments: 5490-v2.txt, HBASE-5490.patch


 The new state that was added  RS_ZK_REGION_FAILED_OPEN was failing the 
 rolling restart.
 So move the new enum to the end of the list.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5441) HRegionThriftServer may not start because of a race-condition


[ 
https://issues.apache.org/jira/browse/HBASE-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219245#comment-13219245
 ] 

Zhihong Yu commented on HBASE-5441:
---

@Scott:
Patch v4 doesn't apply to ThriftServerRunner.java cleanly.
Do you mind regenerating a patch that applies to TRUNK cleanly ?


 HRegionThriftServer may not start because of a race-condition
 -

 Key: HBASE-5441
 URL: https://issues.apache.org/jira/browse/HBASE-5441
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Priority: Minor
 Attachments: HBASE-5441.D1845.1.patch, HBASE-5441.D1845.2.patch, 
 HBASE-5441.D1857.2.patch, HBASE-5441.D1857.3.patch, HBASE-5441.D1857.4.patch


 This happens because the master is not started when ThriftServerRunner tries 
 to create an HBaseAdmin.
 org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not 
 running yet
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)
 at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
 at $Proxy8.getProtocolVersion(Unknown Source)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:108)
 at 
 org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.init(ThriftServerRunner.java:516)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.init(HRegionThriftServer.java:104)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer.init(HRegionThriftServer.java:74)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
 at java.lang.Thread.run(Thread.java:662)
 2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class

2012-02-29 Thread honghua zhu (Created) (JIRA)

Delete the HBaseConfiguration.create for coprocessor.Exec class
---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1


Exec class has a field: private Configuration conf = 
HBaseConfiguration.create()

Client side generates an Exec instance of the class, each initiated Statistics 
request by ExecRPCInvoker
Is so HBaseConfiguration.create for each request needs to call

When the server side deserialize the Exec Called once HBaseConfiguration.create 
in,
HBaseConfiguration.create is a time consuming operation.


private Configuration conf = HBaseConfiguration.create();
This code is only useful for testing code 
(org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
other places with the Exec class, pass a Configuration come,
so no need to conf field a default value.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class


 [ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-5491:
---

Status: Patch Available  (was: Open)

 Delete the HBaseConfiguration.create for coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5488) Fixed OfflineMetaRepair bug


[ 
https://issues.apache.org/jira/browse/HBASE-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219312#comment-13219312
 ] 

stack commented on HBASE-5488:
--

+1

 Fixed OfflineMetaRepair bug 
 

 Key: HBASE-5488
 URL: https://issues.apache.org/jira/browse/HBASE-5488
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
Reporter: gaojinchao
Assignee: gaojinchao
Priority: Minor
 Fix For: 0.90.7, 0.92.1

 Attachments: HBASE-5488-branch92.patch, HBASE-5488-trunk.patch, 
 HBASE-5488_branch90.txt


 I want to use OfflineMetaRepair tools and found onbody fix this bugs. I 
 will make a patch.
  12/01/05 23:23:30 ERROR util.HBaseFsck: Bailed out due to:
  java.lang.IllegalArgumentException: Wrong FS: hdfs:// 
  us01-ciqps1-name01.carrieriq.com:9000/hbase/M2M-INTEGRATION-MM_TION-13
  25190318714/0003d2ede27668737e192d8430dbe5d0/.regioninfo,
  expected: file:///
 at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:352)
 at
  org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
 at
  org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:368)
 at
  org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
 at
  org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:126)
 at
  org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284)
 at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:398)
 at
  org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntry(HBaseFsck.java:256)
 at
  org.apache.hadoop.hbase.util.HBaseFsck.loadTableInfo(HBaseFsck.java:284)
 at
  org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:402)
 at
  org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRe

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class


 [ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-5491:
---

Attachment: HBASE-5491.patch

 Delete the HBaseConfiguration.create for coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219315#comment-13219315
 ] 

stack commented on HBASE-5491:
--

+1 on patch.  Will add comment that setConf is for testing only on commit.   
Waiting on hadoopqa before committing.

 Delete the HBaseConfiguration.create for coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219316#comment-13219316
 ] 

stack commented on HBASE-5491:
--

Although, one question Honghua, why not remove the setConf and in the test do 
new Exec(HBaseConfiguration.create())?

 Delete the HBaseConfiguration.create for coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5489) Add HTable accessor to get regions for a key range

[
https://issues.apache.org/jira/browse/HBASE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219318#comment-13219318
]

stack commented on HBASE-5489:
--

Add HTable accessor to get regions for a key range
--

Key: HBASE-5489
URL: https://issues.apache.org/jira/browse/HBASE-5489
Project: HBase
Issue Type: Improvement
Components: client
Reporter: David S. Wang
Assignee: David S. Wang
Priority: Minor

It would be nice to have an accessor to find all regions that overlap with a
particular range of keys. Right now, the only way to accomplish that is to
call HTable.getStartEndKeys(), then follow that with calls to
getRegionLocation() for the range of keys you are interested in. This
algorithm has 2 drawbacks:
* It returns more keys than is necessary most of the time. This is
especially evident if there are a lot of regions comprising the table and the
range of keys is small.
* It always does a scan of .META. via MetaScannerVisitor for at least
HTable.getStartEndKeys(), and perhaps for HRegionLocations that are not
already cached by the client.
An accessor that limited its scans to a specified range could avoid scanning
.META. at all if the HRegionLocations being fetched were already cached by
the client, thereby potentially making this operation faster in common cases.
Here's a proposal for the accessor:
/**
* Get the corresponding regions for an arbitrary range of keys.
* p
* @param startRow Starting row in range, inclusive
* @param endRow Ending row in range, inclusive
* @return A list of HRegionLocations corresponding to the regions that
* contain the specified range
* @throws IOException if a remote or network exception occurs
*/
public ListHRegionLocation getRegionsInRange(final byte [] startKey,
final byte [] endKey) throws IOException

[jira] [Work started] (HBASE-5489) Add HTable accessor to get regions for a key range

2012-02-29 Thread David S. Wang (Work started) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Work on HBASE-5489 started by David S. Wang.

Add HTable accessor to get regions for a key range
--

Key: HBASE-5489
URL: https://issues.apache.org/jira/browse/HBASE-5489
Project: HBase
Issue Type: Improvement
Components: client
Reporter: David S. Wang
Assignee: David S. Wang
Priority: Minor

[jira] [Updated] (HBASE-5492) Caching StartKeys and EndKeys of Regions


 [ 
https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-5492:
---

Attachment: HBASE-5492.patch

 Caching StartKeys and EndKeys of Regions
 

 Key: HBASE-5492
 URL: https://issues.apache.org/jira/browse/HBASE-5492
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5492.patch


 Each call for HTable.getStartEndKeys will read meta table.
 In particular, 
 in the case of client side multi-threaded concurrency statistics, 
 we must call HTable.coprocessorExec==  getStartKeysInRange == 
 getStartEndKeys,
 resulting in the need to always scan the meta table.
 This is not necessary,
 we can implement the 
 HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) 
 method,
 then, get the StartKeys and EndKeys from the cachedRegionLocations of 
 HConnectionImplementation.
 Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve 
 the performance of statistical

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5492) Caching StartKeys and EndKeys of Regions


 [ 
https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-5492:
---

Status: Patch Available  (was: Open)

 Caching StartKeys and EndKeys of Regions
 

 Key: HBASE-5492
 URL: https://issues.apache.org/jira/browse/HBASE-5492
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5492.patch


 Each call for HTable.getStartEndKeys will read meta table.
 In particular, 
 in the case of client side multi-threaded concurrency statistics, 
 we must call HTable.coprocessorExec==  getStartKeysInRange == 
 getStartEndKeys,
 resulting in the need to always scan the meta table.
 This is not necessary,
 we can implement the 
 HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) 
 method,
 then, get the StartKeys and EndKeys from the cachedRegionLocations of 
 HConnectionImplementation.
 Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve 
 the performance of statistical

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5492) Caching StartKeys and EndKeys of Regions

2012-02-29 Thread honghua zhu (Created) (JIRA)

Caching StartKeys and EndKeys of Regions


 Key: HBASE-5492
 URL: https://issues.apache.org/jira/browse/HBASE-5492
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1
 Attachments: HBASE-5492.patch

Each call for HTable.getStartEndKeys will read meta table.

In particular, 
in the case of client side multi-threaded concurrency statistics, 
we must call HTable.coprocessorExec==  getStartKeysInRange == getStartEndKeys,
resulting in the need to always scan the meta table.

This is not necessary,
we can implement the 
HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) 
method,

then, get the StartKeys and EndKeys from the cachedRegionLocations of 
HConnectionImplementation.

Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the 
performance of statistical


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5492) Caching StartKeys and EndKeys of Regions

2012-02-29 Thread Zhihong Yu (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219333#comment-13219333
 ] 

stack commented on HBASE-5492:
--

Once filled, how does the cache of table locations get refreshed if a table 
region splits?

 Caching StartKeys and EndKeys of Regions
 

 Key: HBASE-5492
 URL: https://issues.apache.org/jira/browse/HBASE-5492
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5492.patch


 Each call for HTable.getStartEndKeys will read meta table.
 In particular, 
 in the case of client side multi-threaded concurrency statistics, 
 we must call HTable.coprocessorExec==  getStartKeysInRange == 
 getStartEndKeys,
 resulting in the need to always scan the meta table.
 This is not necessary,
 we can implement the 
 HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) 
 method,
 then, get the StartKeys and EndKeys from the cachedRegionLocations of 
 HConnectionImplementation.
 Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve 
 the performance of statistical

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class

2012-02-29 Thread honghua zhu (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219334#comment-13219334
 ] 

honghua zhu commented on HBASE-5491:


@stack

I just do not want to repair a lot of code. :)

 Delete the HBaseConfiguration.create for coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219338#comment-13219338
 ] 

Zhihong Yu edited comment on HBASE-5491 at 2/29/12 4:53 PM:


The default Exec ctor is only used by TestCoprocessorEndpoint
I think we shouldn't introduce public method(s) just for unit tests.
Stack's last comment makes sense.

@Honghua:
Can you attach a new patch ?

  was (Author: zhi...@ebaysf.com):
The default Exec ctor is only used by TestCoprocessorEndpoint
I think we shouldn't introduce public method(s) just for unit tests.
Stack's last comment makes sense.

@Honghai:
Can you attach a new patch ?
  
 Delete the HBaseConfiguration.create for coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5491) Delete the HBaseConfiguration.create for coprocessor.Exec class


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219338#comment-13219338
 ] 

Zhihong Yu commented on HBASE-5491:
---

The default Exec ctor is only used by TestCoprocessorEndpoint
I think we shouldn't introduce public method(s) just for unit tests.
Stack's last comment makes sense.

@Honghai:
Can you attach a new patch ?

 Delete the HBaseConfiguration.create for coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5492) Caching StartKeys and EndKeys of Regions

2012-02-29 Thread Hadoop QA (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219345#comment-13219345
]

Hadoop QA commented on HBASE-5492:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12516578/HBASE-5492.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 javadoc. The javadoc tool appears to have generated -131 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 154 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
org.apache.hadoop.hbase.TestRegionRebalancing

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1061//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1061//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1061//console

This message is automatically generated.

Caching StartKeys and EndKeys of Regions

Key: HBASE-5492
URL: https://issues.apache.org/jira/browse/HBASE-5492
Project: HBase
Issue Type: Improvement
Components: client
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.1

Attachments: HBASE-5492.patch

Each call for HTable.getStartEndKeys will read meta table.
In particular,
in the case of client side multi-threaded concurrency statistics,
we must call HTable.coprocessorExec== getStartKeysInRange ==
getStartEndKeys,
resulting in the need to always scan the meta table.
This is not necessary,
we can implement the
HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName)
method,
then, get the StartKeys and EndKeys from the cachedRegionLocations of
HConnectionImplementation.
Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve
the performance of statistical

[jira] [Updated] (HBASE-5491) Remove HBaseConfiguration.create() call from coprocessor.Exec class


 [ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5491:
--

Summary: Remove HBaseConfiguration.create() call from coprocessor.Exec 
class  (was: Delete the HBaseConfiguration.create for coprocessor.Exec class)

 Remove HBaseConfiguration.create() call from coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5491) Remove HBaseConfiguration.create() call from coprocessor.Exec class

2012-02-29 Thread Hadoop QA (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219347#comment-13219347
 ] 

Hadoop QA commented on HBASE-5491:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12516576/HBASE-5491.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -131 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 155 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.replication.TestReplicationPeer
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
  org.apache.hadoop.hbase.io.hfile.TestLruBlockCache
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestImportTsv

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1062//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1062//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1062//console

This message is automatically generated.

 Remove HBaseConfiguration.create() call from coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5270) Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler

[
https://issues.apache.org/jira/browse/HBASE-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219349#comment-13219349
]

stack commented on HBASE-5270:
--

@Prakash

Presume we pass list from splitLogAfterStartup to joinClusters as you suggest
and presume list of servers included the server that had been hosting .META.

Allow that during or just after splitLogAfterStartup, .META. server 'crashes'
-- it becomes unresponsive. Also allow that somehow, just before it hung up,
during a long running log split, .META. took on a couple of edits saying
regions A, B, and C had split.

In assignRootAndMeta, we'll notice the unresponsiveness, force the expiration
of the server that was carrying .META. (this will queue a ServerShutdownHandler
but will not wait on its completion), and we'll then reassign of .META. Its
very likely that .META. will go to one of the other 'good' servers. Its also
likely that the SSH will not have completed its processing before this assign
happens. Thus, on deploy, the .META. will be missing the above A, B, and C
split edits.

Handle potential data loss due to concurrent processing of processFaileOver
and ServerShutdownHandler
-

Key: HBASE-5270
URL: https://issues.apache.org/jira/browse/HBASE-5270
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Zhihong Yu
Assignee: chunhui shen
Fix For: 0.92.1, 0.94.0

Attachments: 5270-90-testcase.patch, 5270-90-testcasev2.patch,
5270-90.patch, 5270-90v2.patch, 5270-90v3.patch, 5270-testcase.patch,
5270-testcasev2.patch, hbase-5270.patch, hbase-5270v2.patch,
hbase-5270v4.patch, hbase-5270v5.patch, hbase-5270v6.patch,
hbase-5270v7.patch, hbase-5270v8.patch, sampletest.txt

This JIRA continues the effort from HBASE-5179. Starting with Stack's
comments about patches for 0.92 and TRUNK:
Reviewing 0.92v17
isDeadServerInProgress is a new public method in ServerManager but it does
not seem to be used anywhere.
Does isDeadRootServerInProgress need to be public? Ditto for meta version.
This method param names are not right 'definitiveRootServer'; what is meant
by definitive? Do they need this qualifier?
Is there anything in place to stop us expiring a server twice if its carrying
root and meta?
What is difference between asking assignment manager isCarryingRoot and this
variable that is passed in? Should be doc'd at least. Ditto for meta.
I think I've asked for this a few times - onlineServers needs to be
explained... either in javadoc or in comment. This is the param passed into
joinCluster. How does it arise? I think I know but am unsure. God love the
poor noob that comes awandering this code trying to make sense of it all.
It looks like we get the list by trawling zk for regionserver znodes that
have not checked in. Don't we do this operation earlier in master setup? Are
we doing it again here?
Though distributed split log is configured, we will do in master single
process splitting under some conditions with this patch. Its not explained in
code why we would do this. Why do we think master log splitting 'high
priority' when it could very well be slower. Should we only go this route if
distributed splitting is not going on. Do we know if concurrent distributed
log splitting and master splitting works?
Why would we have dead servers in progress here in master startup? Because a
servershutdownhandler fired?
This patch is different to the patch for 0.90. Should go into trunk first
with tests, then 0.92. Should it be in this issue? This issue is really hard
to follow now. Maybe this issue is for 0.90.x and new issue for more work on
this trunk patch?
This patch needs to have the v18 differences applied.

[jira] [Updated] (HBASE-5482) In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.


 [ 
https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5482:
--

Attachment: 5482-v2.txt

Patch v2 corrected grammar in comments

 In 0.90, balancer algo leading to same region balanced twice and picking same 
 region with Src and Destination as same RS.
 -

 Key: HBASE-5482
 URL: https://issues.apache.org/jira/browse/HBASE-5482
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.90.7

 Attachments: 5482-v2.txt, HBASE-5482_1.patch


 There are possibility of 2 problems
 - When we populate regionsToMove while iterating the serverinfo in 
 descending manner there is a chance that the same region can be added twice.
 Because in the first loop we do a randomization of the regions.
 Where as when we get we have neededRegions!= 0 we just get the region in the 
 index and add it again . This may lead to have same region in the 
 regionsToMove list.
 - Another problem is 
 when the problem in the first point happens then there is a chance that
 the regionToMove can have the same src and destination and the same region 
 can be picked every 5 mins.
 {code}
 for(Map.EntryHServerInfo, ListHRegionInfo server :
 serversByLoad.descendingMap().entrySet()) {
 BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
 int idx =
   balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
 if (idx = server.getValue().size()) break;
 HRegionInfo region = server.getValue().get(idx);
 if (region.isMetaRegion()) continue; // Don't move meta regions.
 regionsToMove.add(new RegionPlan(region, server.getKey(), null));
 if(--neededRegions == 0) {
   // No more regions needed, done shedding
   break;
 }
   }
 {code}
 If i have meta and root in the top two loaded region server(totally 3 RS), we 
 just skip the regions in those region server and populate the region from the 
 least loaded RS.
 Then in the next loop we iterate from the least loaded server and populate 
 the destination as also the same server.
 This is leading to a condition where every 5 min balancing happens and also 
 the server is same for src and dest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5482) In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.


[ 
https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219352#comment-13219352
 ] 

Zhihong Yu commented on HBASE-5482:
---

Patch looks good.

Please add curly braces around break statement:
{code}
+  if (idx = server.getValue().size())
+break;
{code}
In:
{code}
+  public void testBalanceClusterWithFirstRegionsRootMeta() throws Exception {
{code}
the following should end with 'META are':
{code}
+  // do not randomize in order to create a scenario where ROOT and META is 
{code}
{code}
+  public void testPlanContainsDuplicateRegions() throws Exception {
{code}
Would testNoDuplicateRegionsInPlan() be a good name ?

 In 0.90, balancer algo leading to same region balanced twice and picking same 
 region with Src and Destination as same RS.
 -

 Key: HBASE-5482
 URL: https://issues.apache.org/jira/browse/HBASE-5482
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.90.7

 Attachments: 5482-v2.txt, HBASE-5482_1.patch


 There are possibility of 2 problems
 - When we populate regionsToMove while iterating the serverinfo in 
 descending manner there is a chance that the same region can be added twice.
 Because in the first loop we do a randomization of the regions.
 Where as when we get we have neededRegions!= 0 we just get the region in the 
 index and add it again . This may lead to have same region in the 
 regionsToMove list.
 - Another problem is 
 when the problem in the first point happens then there is a chance that
 the regionToMove can have the same src and destination and the same region 
 can be picked every 5 mins.
 {code}
 for(Map.EntryHServerInfo, ListHRegionInfo server :
 serversByLoad.descendingMap().entrySet()) {
 BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
 int idx =
   balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
 if (idx = server.getValue().size()) break;
 HRegionInfo region = server.getValue().get(idx);
 if (region.isMetaRegion()) continue; // Don't move meta regions.
 regionsToMove.add(new RegionPlan(region, server.getKey(), null));
 if(--neededRegions == 0) {
   // No more regions needed, done shedding
   break;
 }
   }
 {code}
 If i have meta and root in the top two loaded region server(totally 3 RS), we 
 just skip the regions in those region server and populate the region from the 
 least loaded RS.
 Then in the next loop we iterate from the least loaded server and populate 
 the destination as also the same server.
 This is leading to a condition where every 5 min balancing happens and also 
 the server is same for src and dest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5270) Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler

[
https://issues.apache.org/jira/browse/HBASE-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219354#comment-13219354
]

stack commented on HBASE-5270:
--

@Chunhui You have a new method splitLogIfOnline which will split the log if the
server was online. Why do you not expire the server? (You remove the
expireIfOnline method).

Now we have this initializing state, do you think we should also stop the
processing of expired servers during this startup phase and instead queue them
up for processing after the master is up? Could do that in another issue maybe
since this issue has been going on too long and your patch is at least an
improvement on what we currently have (This startup sequence needs a big
refactor IMO -- it is way too complicated figuring the sequence in which stuff
runs).

Are there still holes? For example, say the .META. server crashes AFTER we've
verified it up in assignRootAndMeta but before we get to do a scan of .META. to
rebuild user regions list. Could .META. be assigned w/o log splitting
finishing? (I don't think so... .META. would be offline until the
servershutdown handler ran and it would first split logs).

Good stuff.

Handle potential data loss due to concurrent processing of processFaileOver
and ServerShutdownHandler
-

Key: HBASE-5270
URL: https://issues.apache.org/jira/browse/HBASE-5270
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Zhihong Yu
Assignee: chunhui shen
Fix For: 0.92.1, 0.94.0

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219355#comment-13219355
 ] 

stack commented on HBASE-4991:
--

@Mubarak After looking at FATE and whats involved, I think it a bit much to 
expect that we build that as a prereq. for this facility.  At the same time, 
lets minimize custom code -- code that is particular to the addition of this 
feature only.  Let me do another review of your last patch w/ that in mind.

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5441) HRegionThriftServer may not start because of a race-condition


 [ 
https://issues.apache.org/jira/browse/HBASE-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5441:
--

Attachment: 5441-v5.txt

Patch v5 applies to TRUNK.

 HRegionThriftServer may not start because of a race-condition
 -

 Key: HBASE-5441
 URL: https://issues.apache.org/jira/browse/HBASE-5441
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Priority: Minor
 Attachments: 5441-v5.txt, HBASE-5441.D1845.1.patch, 
 HBASE-5441.D1845.2.patch, HBASE-5441.D1857.2.patch, HBASE-5441.D1857.3.patch, 
 HBASE-5441.D1857.4.patch


 This happens because the master is not started when ThriftServerRunner tries 
 to create an HBaseAdmin.
 org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not 
 running yet
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)
 at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
 at $Proxy8.getProtocolVersion(Unknown Source)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:108)
 at 
 org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.init(ThriftServerRunner.java:516)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.init(HRegionThriftServer.java:104)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer.init(HRegionThriftServer.java:74)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
 at java.lang.Thread.run(Thread.java:662)
 2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5441) HRegionThriftServer may not start because of a race-condition


[ 
https://issues.apache.org/jira/browse/HBASE-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219366#comment-13219366
 ] 

Zhihong Yu commented on HBASE-5441:
---

Patch v5 integrated to TRUNK.

Thanks for the patch Scott.

Thanks for the review Dhruba.

 HRegionThriftServer may not start because of a race-condition
 -

 Key: HBASE-5441
 URL: https://issues.apache.org/jira/browse/HBASE-5441
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Priority: Minor
 Attachments: 5441-v5.txt, HBASE-5441.D1845.1.patch, 
 HBASE-5441.D1845.2.patch, HBASE-5441.D1857.2.patch, HBASE-5441.D1857.3.patch, 
 HBASE-5441.D1857.4.patch


 This happens because the master is not started when ThriftServerRunner tries 
 to create an HBaseAdmin.
 org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not 
 running yet
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)
 at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
 at $Proxy8.getProtocolVersion(Unknown Source)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:108)
 at 
 org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.init(ThriftServerRunner.java:516)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.init(HRegionThriftServer.java:104)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer.init(HRegionThriftServer.java:74)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
 at java.lang.Thread.run(Thread.java:662)
 2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5441) HRegionThriftServer may not start because of a race-condition


 [ 
https://issues.apache.org/jira/browse/HBASE-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5441:
--

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

 HRegionThriftServer may not start because of a race-condition
 -

 Key: HBASE-5441
 URL: https://issues.apache.org/jira/browse/HBASE-5441
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Priority: Minor
 Attachments: 5441-v5.txt, HBASE-5441.D1845.1.patch, 
 HBASE-5441.D1845.2.patch, HBASE-5441.D1857.2.patch, HBASE-5441.D1857.3.patch, 
 HBASE-5441.D1857.4.patch


 This happens because the master is not started when ThriftServerRunner tries 
 to create an HBaseAdmin.
 org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not 
 running yet
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)
 at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
 at $Proxy8.getProtocolVersion(Unknown Source)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:108)
 at 
 org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.init(ThriftServerRunner.java:516)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.init(HRegionThriftServer.java:104)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer.init(HRegionThriftServer.java:74)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
 at java.lang.Thread.run(Thread.java:662)
 2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219375#comment-13219375
 ] 

Zhihong Yu commented on HBASE-4991:
---

@Todd:
Do you agree with what Stack said above ?
If so, I can start reviewing latest patch as well.

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219380#comment-13219380
 ] 

stack commented on HBASE-4991:
--

Reviewing this patch again, could we not obtain this patches's objective with 
merge?  Merge could take a flag which said True/false copy the data from old 
regions into the new merge region





 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5441) HRegionThriftServer may not start because of a race-condition

2012-02-29 Thread dhruba borthakur (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219383#comment-13219383
 ] 

dhruba borthakur commented on HBASE-5441:
-

Thanks for integrating it with trunk and committing it, Ted.

 HRegionThriftServer may not start because of a race-condition
 -

 Key: HBASE-5441
 URL: https://issues.apache.org/jira/browse/HBASE-5441
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Priority: Minor
 Attachments: 5441-v5.txt, HBASE-5441.D1845.1.patch, 
 HBASE-5441.D1845.2.patch, HBASE-5441.D1857.2.patch, HBASE-5441.D1857.3.patch, 
 HBASE-5441.D1857.4.patch


 This happens because the master is not started when ThriftServerRunner tries 
 to create an HBaseAdmin.
 org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not 
 running yet
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)
 at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
 at $Proxy8.getProtocolVersion(Unknown Source)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:108)
 at 
 org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.init(ThriftServerRunner.java:516)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.init(HRegionThriftServer.java:104)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer.init(HRegionThriftServer.java:74)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
 at java.lang.Thread.run(Thread.java:662)
 2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219395#comment-13219395
 ] 

Zhihong Yu commented on HBASE-4991:
---

Are we going to enhance Merge by allowing to discard data belonging to one of 
the regions ?
How should we deal with various failure scenarios in the process of merging ?

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region

[
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219412#comment-13219412
]

stack commented on HBASE-4991:
--

After looking again too at the patch, it has too much custom code that is all
about region delete.

It should take a range as Todd suggests earlier rather than list of regions.
This means you can not pass a list of discontinuous regions but thats ok I
think; just do multiple invocations.

This seems to have wrong param name and javadoc:

{code}
+ /**
+ * Gets the count of online regions of the table in a region server.
+ * This method looks at the in-memory onlineRegions.
+ * @param regionName
+ * @return int regions count
+ * @throws IOException
+ */
+ public int getRegionsCount(byte[] regionName) throws IOException;
{code}

When I see MasterDeleteRegionTracker, and the equivalent for regionservers, it
makes me yearn for a generic framework that these things could run on; it
strikes me as too much custom code and custom handlers. This we should fix.
We should come up w/ generics that can be customized to do feature specifics.

Why are we using a janitor to do the delete of regions rather than an executor?

Why we have this getDeleteRegionTracker ?

The generic soln Interface would have a method the balancer would check...

+ if (deleteRegionTracker.isDeleteRegionInProgress()) {

Rather than do the above for every feature we add.

Should this getDeleteRegionStatusFromDeleteRegionTracker be in the
DeleteRegionTracker? And should it be something that is apart from the Master
rather than in the master?

This seems wrong: getDeleteRegionTracker in the MasterServices Interface.

Why we add it there? Why can't it be independent of Master? Having to have a
Master makes it harder to test I'm sure.

DeleteRegionHandler should not be dealing w/ balancer. That seems dirty.

This seems racy: waitForInflightSplit

Do we do this every time? +moveStoreFilesToNewRegionDir(byFamily, fs,
tableDir, newRegionInfo);

If so, is this actually a merge and not a delete?

Do these methods need to be in HREgionInfo?

moveDataFromAdjacentRegionToNewRegion
createNewRegionFromAdjacentRegion

Could be in HRegion or in a RegionUtil class? RS is already bloated.

A bunch of these other methods ... adding new region and deleting old region
... would seem to have overlap with existing code where we add regions to meta
after open and also w/ merge code?

We can't have master package refernced in zookeeper package; i.e. see
MasterDeleteRegionTracker.

I've already commented on other stuff in this patch.

In general the patch is well done. It just adds a bunch of custom facility w/o
genericizing at least some aspects so could be used by other features yet to
come. In particular, this looks to be a specialization on merge. If so, lets
go for merge altogether.

Provide capability to delete named region
-

Key: HBASE-4991
URL: https://issues.apache.org/jira/browse/HBASE-4991
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
Fix For: 0.94.0

Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch

See discussion titled 'Able to control routing to Solr shards or not' on
lily-discuss
User may want to quickly dispose of out of date records by deleting specific
regions.

[jira] [Commented] (HBASE-5270) Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler

2012-02-29 Thread Prakash Khemani (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219413#comment-13219413
]

Prakash Khemani commented on HBASE-5270:

@Stack

If we presume that the list of servers that joinClusters received contains
the server hosting .META., then the next step, that you outlined in your
scenario, cannot be allowed. If we are splitting logs for .META. then we
have determined that meta-server was not running and therefore it cannot
be taking edits. The problem you are outlining is probably still there but
the scenario has to be refined.

Anyway my point was - at startup master should determine once what servers
are up and what are not. This should include whether ROOT and META are
assigned or not. And then it should initialize everything based on that
knowledge which must not change during initialization. Anything that
changes during initialization should be taken care of by the normal
Server-handlers. But I have to admit, I don't understand the assignment
complexities very well Š I will read up some more.

Handle potential data loss due to concurrent processing of processFaileOver
and ServerShutdownHandler
-

Key: HBASE-5270
URL: https://issues.apache.org/jira/browse/HBASE-5270
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Zhihong Yu
Assignee: chunhui shen
Fix For: 0.92.1, 0.94.0

[jira] [Commented] (HBASE-4991) Provide capability to delete named region

[
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219421#comment-13219421
]

stack commented on HBASE-4991:
--

bq. Are we going to enhance Merge by allowing to discard data belonging to one
of the regions ?

This feature looks to be adding online merge to me. I need clarification from
Mubarak that that is indeed the case. If so, this issue is mislabeled and the
patch needs redoing.

I was just suggesting that if you want to actually drop a regions data, you
could pass a flag to merge and it would not bother copying over the files from
old regions. That would be an option. This patch as is does not do that. It
seems to copy old region data into new regions. Was just a suggestion.

bq. How should we deal with various failure scenarios in the process of merging
?

Eh... in a manner which is resilient against failures, TBD. I don't get your
question Ted. Are you asking me or the Author of this patch?

Provide capability to delete named region
-

Key: HBASE-4991
URL: https://issues.apache.org/jira/browse/HBASE-4991
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
Fix For: 0.94.0

Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch

See discussion titled 'Able to control routing to Solr shards or not' on
lily-discuss
User may want to quickly dispose of out of date records by deleting specific
regions.

[jira] [Commented] (HBASE-5270) Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler

[
https://issues.apache.org/jira/browse/HBASE-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219428#comment-13219428
]

stack commented on HBASE-5270:
--

@Prakash

bq. then the next step, that you outlined in your scenario, cannot be
allowed

How should we do this boss?

bq. The problem you are outlining is probably still there but the scenario has
to be refined.

What should I add? If we allow that the split could take a long time, its
possible that on entry to the log splitting the server was good but by the end
it could have gone AWOL.

bq. And then it should initialize everything based on that knowledge which must
not change during initialization.

I think the root issue is that it needs to scan .META. and -ROOT- as part of
the startup; they need to be assigned and up w/ all edits in place. Thats
whats proving to be a little tough to ensure.

(Thanks for the review P).

Handle potential data loss due to concurrent processing of processFaileOver
and ServerShutdownHandler
-

Key: HBASE-5270
URL: https://issues.apache.org/jira/browse/HBASE-5270
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Zhihong Yu
Assignee: chunhui shen
Fix For: 0.92.1, 0.94.0

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219431#comment-13219431
 ] 

Zhihong Yu commented on HBASE-4991:
---

For:
{code}
+  public int getRegionsCount(byte[] regionName) throws IOException;
{code}
regionName is used to extract table name. We should rename the method to, e.g. 
getRegionsCountForTable().

w.r.t. utilizing code from Merge, I looked at Merge.mergeTwoRegions() and saw 
no fault handling code.
I think if we add fault-tolerant code to Merge, it might look similar to what 
Mubarak has now.

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219436#comment-13219436
 ] 

Mubarak Seyed commented on HBASE-4991:
--

We are going to remove
{code}
public int getRegionsCount(byte[] regionName) throws IOException
{code}
as we will use getOnlineRegions() and process them in client-side, please refer 
my comment @ 24/Feb/12 00:16

Will read the onlineMerge code.

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5491) Remove HBaseConfiguration.create() call from coprocessor.Exec class

2012-02-29 Thread Andrew Purtell (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219470#comment-13219470
 ] 

Andrew Purtell commented on HBASE-5491:
---

+1

 Remove HBaseConfiguration.create() call from coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219477#comment-13219477
 ] 

Mubarak Seyed commented on HBASE-4991:
--

@Stack
bq.This feature looks to be adding online merge to me. I need clarification 
from Mubarak that that is indeed the case. If so, this issue is mislabeled and 
the patch needs redoing.
Online merge requires table needs to be disabled but deleteRegion (deleteRange) 
does not require table needs to be disabled.
We were discussing about making use of HMerge.merge() (please refer my comment 
@ 27/Jan/12 06:36) but it checks for half of region-size

{code}
this.maxFilesize = conf.getLong(HConstants.HREGION_MAX_FILESIZE,
  HConstants.DEFAULT_MAX_FILE_SIZE);
..
if ((currentSize + nextSize) = (maxFilesize / 2)) {
  // We merge two adjacent regions if their total size is less than
  // one half of the desired maximum size
{code}
so, in this case an adjacent region size can be  half of max of region-size? 






 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region

[
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219511#comment-13219511
]

stack commented on HBASE-4991:
--

bq. Online merge requires table needs to be disabled but deleteRegion
(deleteRange) does not require table needs to be disabled.

We've had ongoing conversation -- before your time so you are not expected to
have known about it -- on our doing an online merge. Its actually pretty
critical need. See HBASE-1621 for some history (Ignore its title where it
says table should be offline -- it should be online).

FYI, the current merge code is broke and unused. It works for a unit test but
I'd say its years since anyone tried to use it to actually do anything useful.

So, we are agreed that conceptually, whats going on here is region merging? If
so, that helps understanding around whats going on here. We should also likely
rename what this issue does to be about merging since thats how we've been
describing this operation over the years.

Provide capability to delete named region
-

Key: HBASE-4991
URL: https://issues.apache.org/jira/browse/HBASE-4991
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
Fix For: 0.94.0

Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch

See discussion titled 'Able to control routing to Solr shards or not' on
lily-discuss
User may want to quickly dispose of out of date records by deleting specific
regions.

[jira] [Commented] (HBASE-5491) Remove HBaseConfiguration.create() call from coprocessor.Exec class


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219527#comment-13219527
 ] 

stack commented on HBASE-5491:
--

@Honghua My suggestion would make your patch smaller.

 Remove HBaseConfiguration.create() call from coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2818) Cannot force a region to close when it has no RS entry in META

2012-02-29 Thread stack (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-2818:
-

 Priority: Critical  (was: Major)
Affects Version/s: (was: 0.90.0)
   0.94.0
   0.92.1

Marking critical for 0.94.0 and for 0.92.1.  Need this doing fixup.

 Cannot force a region to close when it has no RS entry in META
 --

 Key: HBASE-2818
 URL: https://issues.apache.org/jira/browse/HBASE-2818
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.1, 0.94.0
Reporter: Todd Lipcon
Priority: Critical

 I have a region that's open on a server, but META thinks it's not deployed 
 anywhere. I get the following when trying to close it:
 hbase(main):002:0 close_region 
 'usertable,user302806495,1278457018956.c4ad0681f7be3995490c745861af66ea.', 
 '192.168.42.41:60020'
 ERROR: java.io.IOException: java.io.IOException: 
 java.lang.NullPointerException
 at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:479)
 at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:453)
 at 
 org.apache.hadoop.hbase.master.HMaster.modifyTable(HMaster.java:1021)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5493) [book] External apis / Thrift - adding link to Thrift javadoc.

2012-02-29 Thread Doug Meil (Created) (JIRA)

[book] External apis / Thrift - adding link to Thrift javadoc.
--

 Key: HBASE-5493
 URL: https://issues.apache.org/jira/browse/HBASE-5493
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor


external_apis.xml
* Thrift:  adding link to Thrift javadoc.  This came up on the dist-list 
recently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5493) [book] External apis / Thrift - adding link to Thrift javadoc.


 [ 
https://issues.apache.org/jira/browse/HBASE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5493:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] External apis / Thrift - adding link to Thrift javadoc.
 --

 Key: HBASE-5493
 URL: https://issues.apache.org/jira/browse/HBASE-5493
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: external_apis_hbase_5493.xml.patch


 external_apis.xml
 * Thrift:  adding link to Thrift javadoc.  This came up on the dist-list 
 recently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5493) [book] External apis / Thrift - adding link to Thrift javadoc.


 [ 
https://issues.apache.org/jira/browse/HBASE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5493:
-

Status: Patch Available  (was: Open)

 [book] External apis / Thrift - adding link to Thrift javadoc.
 --

 Key: HBASE-5493
 URL: https://issues.apache.org/jira/browse/HBASE-5493
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: external_apis_hbase_5493.xml.patch


 external_apis.xml
 * Thrift:  adding link to Thrift javadoc.  This came up on the dist-list 
 recently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region

2012-02-29 Thread Lars Hofhansl (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219568#comment-13219568
]

Lars Hofhansl commented on HBASE-4991:
--

I have not been following this too closely, so pardon my ignorance... But it
seems we're going off in a tangent.

This is not the same as merge, right? The region's data will be gone, we just
need to plug the .META. hole (hbck, could even do that for us).
Or maybe we don't even touch .META. and just delete the DFS files.

1. close/unassign the region
2. remove the region's files
3. either
o re-open the region, i.e. keep the .META. entry for the empty region, and
merge later when convenient
or
o remove the region's .META. entry
4. done :)

Maybe I'm just talking nonsense, but does it have to be much more complicated
than this?

Provide capability to delete named region
-

Key: HBASE-4991
URL: https://issues.apache.org/jira/browse/HBASE-4991
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
Fix For: 0.94.0

Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch

See discussion titled 'Able to control routing to Solr shards or not' on
lily-discuss
User may want to quickly dispose of out of date records by deleting specific
regions.

[jira] [Created] (HBASE-5494) Introduce a zk hosted table-wide read/write lock so only one table operation at a time

2012-02-29 Thread stack (Created) (JIRA)

Introduce a zk hosted table-wide read/write lock so only one table operation at 
a time
--

 Key: HBASE-5494
 URL: https://issues.apache.org/jira/browse/HBASE-5494
 Project: HBase
  Issue Type: Improvement
Reporter: stack


I saw this facility over in the accumulo code base.

Currently we just try to sort out the mess when splits come in during an online 
schema edit; somehow we figure we can figure all possible region transition 
combinations and make the right call.

We could try and narrow the number of combinations by taking out a zk table 
lock when doing table operations.

For example, on split or merge, we could take a read-only lock meaning the 
table can't be disabled while these are running.

We could then take a write only lock if we want to ensure the table doesn't 
change while disabling or enabling process is happening.

Shouldn't be too hard to add.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5493) [book] External apis / Thrift - adding link to Thrift javadoc.


 [ 
https://issues.apache.org/jira/browse/HBASE-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5493:
-

Attachment: external_apis_hbase_5493.xml.patch

 [book] External apis / Thrift - adding link to Thrift javadoc.
 --

 Key: HBASE-5493
 URL: https://issues.apache.org/jira/browse/HBASE-5493
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: external_apis_hbase_5493.xml.patch


 external_apis.xml
 * Thrift:  adding link to Thrift javadoc.  This came up on the dist-list 
 recently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5495) [book] adding a Faq in the General section for examples

2012-02-29 Thread Doug Meil (Created) (JIRA)

[book] adding a Faq in the General section for examples
-

 Key: HBASE-5495
 URL: https://issues.apache.org/jira/browse/HBASE-5495
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil


book.xml
* adding a FAQ entry in the General section for NoSQL/HBase examples.  This 
just links to the already existing more information appendix entry which has 
a list of videos, papers, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5495) [book] adding a Faq in the General section for examples


 [ 
https://issues.apache.org/jira/browse/HBASE-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5495:
-

Attachment: book_hbase_5495.xml.patch

 [book] adding a Faq in the General section for examples
 -

 Key: HBASE-5495
 URL: https://issues.apache.org/jira/browse/HBASE-5495
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: book_hbase_5495.xml.patch


 book.xml
 * adding a FAQ entry in the General section for NoSQL/HBase examples.  This 
 just links to the already existing more information appendix entry which 
 has a list of videos, papers, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5495) [book] adding a Faq in the General section for examples


 [ 
https://issues.apache.org/jira/browse/HBASE-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5495:
-

Status: Patch Available  (was: Open)

 [book] adding a Faq in the General section for examples
 -

 Key: HBASE-5495
 URL: https://issues.apache.org/jira/browse/HBASE-5495
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: book_hbase_5495.xml.patch


 book.xml
 * adding a FAQ entry in the General section for NoSQL/HBase examples.  This 
 just links to the already existing more information appendix entry which 
 has a list of videos, papers, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5495) [book] adding a Faq in the General section for examples


 [ 
https://issues.apache.org/jira/browse/HBASE-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5495:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] adding a Faq in the General section for examples
 -

 Key: HBASE-5495
 URL: https://issues.apache.org/jira/browse/HBASE-5495
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: book_hbase_5495.xml.patch


 book.xml
 * adding a FAQ entry in the General section for NoSQL/HBase examples.  This 
 just links to the already existing more information appendix entry which 
 has a list of videos, papers, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219575#comment-13219575
 ] 

Mubarak Seyed commented on HBASE-4991:
--

bq.So, we are agreed that conceptually, whats going on here is region merging?
Merge with two-regions-data Vs Merge-with-one-region-data

We do merge just to cover META hole, isn't? IMO deleteRange is a task/function 
from cmdLine/API and we do use merge for an implementation (as part of the task)

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5496) [book] ops_mgt.xml - filling in HBase Monitoring section overview

2012-02-29 Thread Doug Meil (Created) (JIRA)

[book] ops_mgt.xml - filling in HBase Monitoring section overview
-

 Key: HBASE-5496
 URL: https://issues.apache.org/jira/browse/HBASE-5496
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil


ops_mgt.xml
* filling in HBase Monitoring section overview, which previously didn't have 
anything
* Used a dist-list email from JD as a starting point for macro monitoring

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5496) [book] ops_mgt.xml - filling in HBase Monitoring section overview


 [ 
https://issues.apache.org/jira/browse/HBASE-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5496:
-

Status: Patch Available  (was: Open)

 [book] ops_mgt.xml - filling in HBase Monitoring section overview
 -

 Key: HBASE-5496
 URL: https://issues.apache.org/jira/browse/HBASE-5496
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: ops_mgt_hbase_5496.xml.patch


 ops_mgt.xml
 * filling in HBase Monitoring section overview, which previously didn't have 
 anything
 * Used a dist-list email from JD as a starting point for macro monitoring

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5496) [book] ops_mgt.xml - filling in HBase Monitoring section overview


 [ 
https://issues.apache.org/jira/browse/HBASE-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5496:
-

Attachment: ops_mgt_hbase_5496.xml.patch

 [book] ops_mgt.xml - filling in HBase Monitoring section overview
 -

 Key: HBASE-5496
 URL: https://issues.apache.org/jira/browse/HBASE-5496
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: ops_mgt_hbase_5496.xml.patch


 ops_mgt.xml
 * filling in HBase Monitoring section overview, which previously didn't have 
 anything
 * Used a dist-list email from JD as a starting point for macro monitoring

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5496) [book] ops_mgt.xml - filling in HBase Monitoring section overview


 [ 
https://issues.apache.org/jira/browse/HBASE-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5496:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] ops_mgt.xml - filling in HBase Monitoring section overview
 -

 Key: HBASE-5496
 URL: https://issues.apache.org/jira/browse/HBASE-5496
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: ops_mgt_hbase_5496.xml.patch


 ops_mgt.xml
 * filling in HBase Monitoring section overview, which previously didn't have 
 anything
 * Used a dist-list email from JD as a starting point for macro monitoring

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5497) Add protobuf as M/R dependency jar (mapred)

2012-02-29 Thread Lars Hofhansl (Created) (JIRA)

Add protobuf as M/R dependency jar (mapred)
---

 Key: HBASE-5497
 URL: https://issues.apache.org/jira/browse/HBASE-5497
 Project: HBase
  Issue Type: Sub-task
  Components: mapreduce
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0


Getting this from M/R jobs (Export for example):

Error: java.lang.ClassNotFoundException: com.google.protobuf.Message
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at 
org.apache.hadoop.hbase.io.HbaseObjectWritable.clinit(HbaseObjectWritable.java:262)


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5497) Add protobuf as M/R dependency jar (mapred)

2012-02-29 Thread Lars Hofhansl (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl resolved HBASE-5497.
--

  Resolution: Fixed
Hadoop Flags:   (was: Reviewed)

Committed to 0.94 and trunk (exact same change as HBASE-5460, but for mapred)

 Add protobuf as M/R dependency jar (mapred)
 ---

 Key: HBASE-5497
 URL: https://issues.apache.org/jira/browse/HBASE-5497
 Project: HBase
  Issue Type: Sub-task
  Components: mapreduce
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0


 Getting this from M/R jobs (Export for example):
 Error: java.lang.ClassNotFoundException: com.google.protobuf.Message
 at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.clinit(HbaseObjectWritable.java:262)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region

[
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219638#comment-13219638
]

stack commented on HBASE-4991:
--

@Lars

bq. This is not the same as merge, right?

Sounds like it is.

bq. The region's data will be gone

This patch seems to copy the data from the deleted region up into the new
hole-plugging region. It doesn't seem to delete it.

As to your 1., 2., 3... yes, thats what this patch does only the operators and
the classes are all named DeleteRegion* blah when what is happening is region
merging.

I think its important to get the concept right else its going to confuse for
ever after.

@Mubarak So, sounds like the command/api could also be named merge rather than
deleteRegion (You are not actually deleting the data, is that right?)?

Provide capability to delete named region
-

Key: HBASE-4991
URL: https://issues.apache.org/jira/browse/HBASE-4991
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
Fix For: 0.94.0

Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch

See discussion titled 'Able to control routing to Solr shards or not' on
lily-discuss
User may want to quickly dispose of out of date records by deleting specific
regions.

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219646#comment-13219646
 ] 

Zhihong Yu commented on HBASE-4991:
---

The data for specified region is deleted. See the following in 
HRegionServer.java:
{code}
+ // delete the dest region in FS
+ deleteRegionFromFs(regionName,
+ hRegion,
+ RegionDeletionStatus.RegionDeletionState.DEST_REGION_DELETION_FROM_FS,
+ 
RegionDeletionStatus.RegionDeletionState.DEST_REGION_DELETION_FROM_FS_FAILED);
{code}

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5441) HRegionThriftServer may not start because of a race-condition

2012-02-29 Thread Scott Chen (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219647#comment-13219647
 ] 

Scott Chen commented on HBASE-5441:
---

Thanks for the review and commit, Ted.

 HRegionThriftServer may not start because of a race-condition
 -

 Key: HBASE-5441
 URL: https://issues.apache.org/jira/browse/HBASE-5441
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Priority: Minor
 Attachments: 5441-v5.txt, HBASE-5441.D1845.1.patch, 
 HBASE-5441.D1845.2.patch, HBASE-5441.D1857.2.patch, HBASE-5441.D1857.3.patch, 
 HBASE-5441.D1857.4.patch


 This happens because the master is not started when ThriftServerRunner tries 
 to create an HBaseAdmin.
 org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not 
 running yet
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)
 at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
 at $Proxy8.getProtocolVersion(Unknown Source)
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
 at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:108)
 at 
 org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.init(ThriftServerRunner.java:516)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.init(HRegionThriftServer.java:104)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionThriftServer.init(HRegionThriftServer.java:74)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
 at java.lang.Thread.run(Thread.java:662)
 2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219649#comment-13219649
 ] 

Mubarak Seyed commented on HBASE-4991:
--

@Stack
bq. sounds like the command/api could also be named merge rather than 
deleteRegion (You are not actually deleting the data, is that right?)?
We do delete the data of region (to be deleted) The real confusion with the 
terminology here is that merge makes 2 or more regions into one with data 
remains same (but the start/end keys are different after the merge) but 
deleteRegion makes one region (or multiple regions) data will be deleted from 
HDFS (and make a single region in meta with modified start/end key).

For example:
We have three regions namely R1, R2, and R3

When we do merge R2 with R3 - new region would be (R2R3)' and data of R2 and 
R3 will be moved to (R2R3)'
so, in META it would look like

R1 - start/end key, location
(R2R3)' - modified start/end key, location
r4 - start/end key, location

When we do deleteRange of single region (R2) - new region would be (R1)', 
meaning R1 end key be R2s end key, data of R2 will be deleted, and R1 data will 
be merged with R1'
so, in META it would look like

(R1)' - modified start/end key, location
R3 - start/end key, location

Make sense?

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5418) use different memstoreTS for different operations in the same RowMutation.

2012-02-29 Thread Phabricator (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219655#comment-13219655
 ] 

Phabricator commented on HBASE-5418:


aaiyer has commented on the revision HBASE-5418 [jira] use different 
memstoreTS for different operations in the same RowMutation..

INLINE COMMENTS
  
src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConsistencyControl.java:121
 I'll do that.

REVISION DETAIL
  https://reviews.facebook.net/D1761


 use different memstoreTS for different operations in the same RowMutation.
 --

 Key: HBASE-5418
 URL: https://issues.apache.org/jira/browse/HBASE-5418
 Project: HBase
  Issue Type: Sub-task
  Components: client, coprocessors, regionserver
Reporter: Amitanand Aiyer
Assignee: Amitanand Aiyer
 Fix For: 0.94.0

 Attachments: HBASE-5418.D1761.1.patch


 Assigning different memstoreTS will enable us to guarantee that the
 operations will appear to take effect, along the same order, in which
 they were added to create the RowMutation.
 Based on the diff after renaming to RowMutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5418) use different memstoreTS for different operations in the same RowMutation.

2012-02-29 Thread Phabricator (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219654#comment-13219654
 ] 

Phabricator commented on HBASE-5418:


aaiyer has commented on the revision HBASE-5418 [jira] use different 
memstoreTS for different operations in the same RowMutation..

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:4245 I think 
we should completeMemstoreInsert for all the writeEntries, regardless of 
weather they suceed or not.

  If we do not call completeMemstoreInsert (for a incomplete operation), then 
none of the puts/deletes that start after that operation can ever complete.

  As long as we are totally giving up on retrying the operation, technically, 
it should be fine to call completeMemstoreInsert. All it does, as far as I can 
tell, is to remove the writeEntry from the list of current writes.

  (Although, from code inspection, it seems unlikely why one will suceed and 
the other does not --- because applyFamilyMapToMemstore does not seem to throw 
exceptions).

REVISION DETAIL
  https://reviews.facebook.net/D1761


 use different memstoreTS for different operations in the same RowMutation.
 --

 Key: HBASE-5418
 URL: https://issues.apache.org/jira/browse/HBASE-5418
 Project: HBase
  Issue Type: Sub-task
  Components: client, coprocessors, regionserver
Reporter: Amitanand Aiyer
Assignee: Amitanand Aiyer
 Fix For: 0.94.0

 Attachments: HBASE-5418.D1761.1.patch


 Assigning different memstoreTS will enable us to guarantee that the
 operations will appear to take effect, along the same order, in which
 they were added to create the RowMutation.
 Based on the diff after renaming to RowMutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5497) Add protobuf as M/R dependency jar (mapred)

2012-02-29 Thread Hudson (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219658#comment-13219658
 ] 

Hudson commented on HBASE-5497:
---

Integrated in HBase-0.94 #4 (See [https://builds.apache.org/job/HBase-0.94/4/])
HBASE-5497 Add protobuf as M/R dependency jar (mapred) (Revision 1295328)

 Result = SUCCESS
larsh : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java


 Add protobuf as M/R dependency jar (mapred)
 ---

 Key: HBASE-5497
 URL: https://issues.apache.org/jira/browse/HBASE-5497
 Project: HBase
  Issue Type: Sub-task
  Components: mapreduce
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0


 Getting this from M/R jobs (Export for example):
 Error: java.lang.ClassNotFoundException: com.google.protobuf.Message
 at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.clinit(HbaseObjectWritable.java:262)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5498) S

2012-02-29 Thread Francis Liu (Created) (JIRA)

S
-

 Key: HBASE-5498
 URL: https://issues.apache.org/jira/browse/HBASE-5498
 Project: HBase
  Issue Type: Improvement
Reporter: Francis Liu




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219678#comment-13219678
 ] 

stack commented on HBASE-4991:
--

Implementation-wise, this is a merge with the added option that we not copy the 
data of the regions we are merging.

 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-4688) Make it so can run PE w/o having to put hbase jar on CLASSPATH

2012-02-29 Thread Todd Lipcon (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon reassigned HBASE-4688:
--

Assignee: stack

 Make it so can run PE w/o having to put hbase jar on CLASSPATH
 --

 Key: HBASE-4688
 URL: https://issues.apache.org/jira/browse/HBASE-4688
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.92.0

 Attachments: pe.txt


 I need this:
 {code}
 diff --git a/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java 
 b/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
 index 3982eff..ef47d0d 100644
 --- a/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
 +++ b/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
 @@ -570,6 +570,9 @@ public class PerformanceEvaluation {
  TextOutputFormat.setOutputPath(job, new Path(inputDir,outputs));
  
  TableMapReduceUtil.addDependencyJars(job);
 +// Add a Class from the hbase.jar so it gets registered too.
 +TableMapReduceUtil.addDependencyJars(job.getConfiguration(),
 +  org.apache.hadoop.hbase.util.Bytes.class);
  job.waitForCompletion(true);
}
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5399) Cut the link between the client and the zookeeper ensemble

2012-02-29 Thread nkeywal (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219681#comment-13219681
 ] 

nkeywal commented on HBASE-5399:


v16 (still in progress, some unit tests fail, indent  comments to redo  so 
on), after a discussion with Stack.

HConnection is a connection to the cluster.
However, the fact that the cluster is composed of master, zookeeper and region 
servers should be hidden from the client: some functions need a master some 
others not. This is not the client problem. Especially, these functions can 
move (from example, getting a table descriptor, today a master function, could 
become a region server function). So the getMaster  getZookeeperWatcher, 
shared or not, should not be in HConnection interface.

Client functions are today split in two classes HBaseAdmin  HConnection (with 
stuff like processBatch, listTables, getHTableDescriptor). It could make sense 
to split HConnection further, but anyway we already have two classes using 
master, and the master connection should remain shared between these two 
classes. This should be handled by the HConnection as it is its core 
responsibility and it's as well much simpler technically. So we need to have 
package visible function to get them for HBaseAdmin. I prefer to have them in 
HConnectionImplementation only, even it it implies a cast in HBaseAdmin, as 
this makes HConnection clean from a client point of view.

We stick to the keep alive mechanism with the Closeable, and hence a dynamic 
proxy for master and a subclass for ZooKeeperMaster.

Note that if master  zookeeper are implementation details, the same should 
apply to HRegionInterface (HConnection#getHRegionConnection). But there are 
dependencies from the master package so it can not be included in this jira. 
The keep alive mechanism could be applied to HRegionInterface as well.

 Cut the link between the client and the zookeeper ensemble
 --

 Key: HBASE-5399
 URL: https://issues.apache.org/jira/browse/HBASE-5399
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.94.0
 Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5399_inprogress.patch, 5399_inprogress.v14.patch, 
 5399_inprogress.v16.patch, 5399_inprogress.v3.patch, 5399_inprogress.v9.patch


 The link is often considered as an issue, for various reasons. One of them 
 being that there is a limit on the number of connection that ZK can manage. 
 Stack was suggesting as well to remove the link to master from HConnection.
 There are choices to be made considering the existing API (that we don't want 
 to break).
 The first patches I will submit on hadoop-qa should not be committed: they 
 are here to show the progress on the direction taken.
 ZooKeeper is used for:
 - public getter, to let the client do whatever he wants, and close ZooKeeper 
 when closing the connection = we have to deprecate this but keep it.
 - read get master address to create a master = now done with a temporary 
 zookeeper connection
 - read root location = now done with a temporary zookeeper connection, but 
 questionable. Used in public function locateRegion. To be reworked.
 - read cluster id = now done once with a temporary zookeeper connection.
 - check if base done is available = now done once with a zookeeper 
 connection given as a parameter
 - isTableDisabled/isTableAvailable = public functions, now done with a 
 temporary zookeeper connection.
  - Called internally from HBaseAdmin and HTable
 - getCurrentNrHRS(): public function to get the number of region servers and 
 create a pool of thread = now done with a temporary zookeeper connection
 -
 Master is used for:
 - getMaster public getter, as for ZooKeeper = we have to deprecate this but 
 keep it.
 - isMasterRunning(): public function, used internally by HMerge  HBaseAdmin
 - getHTableDescriptor*: public functions offering access to the master.  = 
 we could make them using a temporary master connection as well.
 Main points are:
 - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a 
 strongly coupled architecture ;-). This can be changed, but requires a lot of 
 modifications in these classes (likely adding a class in the middle of the 
 hierarchy, something like that). Anyway, non connected client will always be 
 really slower, because it's a tcp connection, and establishing a tcp 
 connection is slow.
 - having a link between ZK and all the client seems to make sense for some 
 Use Cases. However, it won't scale if a TCP connection is required for every 
 client
 - if we move the table descriptor part away from the client, we need to find 
 a new place for it.
 - we will have the same issue if HBaseAdmin

[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble

2012-02-29 Thread nkeywal (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5399:
---

Attachment: 5399_inprogress.v16.patch

 Cut the link between the client and the zookeeper ensemble
 --

 Key: HBASE-5399
 URL: https://issues.apache.org/jira/browse/HBASE-5399
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.94.0
 Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5399_inprogress.patch, 5399_inprogress.v14.patch, 
 5399_inprogress.v16.patch, 5399_inprogress.v3.patch, 5399_inprogress.v9.patch


 The link is often considered as an issue, for various reasons. One of them 
 being that there is a limit on the number of connection that ZK can manage. 
 Stack was suggesting as well to remove the link to master from HConnection.
 There are choices to be made considering the existing API (that we don't want 
 to break).
 The first patches I will submit on hadoop-qa should not be committed: they 
 are here to show the progress on the direction taken.
 ZooKeeper is used for:
 - public getter, to let the client do whatever he wants, and close ZooKeeper 
 when closing the connection = we have to deprecate this but keep it.
 - read get master address to create a master = now done with a temporary 
 zookeeper connection
 - read root location = now done with a temporary zookeeper connection, but 
 questionable. Used in public function locateRegion. To be reworked.
 - read cluster id = now done once with a temporary zookeeper connection.
 - check if base done is available = now done once with a zookeeper 
 connection given as a parameter
 - isTableDisabled/isTableAvailable = public functions, now done with a 
 temporary zookeeper connection.
  - Called internally from HBaseAdmin and HTable
 - getCurrentNrHRS(): public function to get the number of region servers and 
 create a pool of thread = now done with a temporary zookeeper connection
 -
 Master is used for:
 - getMaster public getter, as for ZooKeeper = we have to deprecate this but 
 keep it.
 - isMasterRunning(): public function, used internally by HMerge  HBaseAdmin
 - getHTableDescriptor*: public functions offering access to the master.  = 
 we could make them using a temporary master connection as well.
 Main points are:
 - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a 
 strongly coupled architecture ;-). This can be changed, but requires a lot of 
 modifications in these classes (likely adding a class in the middle of the 
 hierarchy, something like that). Anyway, non connected client will always be 
 really slower, because it's a tcp connection, and establishing a tcp 
 connection is slow.
 - having a link between ZK and all the client seems to make sense for some 
 Use Cases. However, it won't scale if a TCP connection is required for every 
 client
 - if we move the table descriptor part away from the client, we need to find 
 a new place for it.
 - we will have the same issue if HBaseAdmin (for both ZK  Master), may be we 
 can put a timeout on the connection. That would make the whole system less 
 deterministic however.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4991) Provide capability to delete named region

2012-02-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219685#comment-13219685
 ] 

Mubarak Seyed commented on HBASE-4991:
--

bq. Implementation-wise, this is a merge with the added option that we not copy 
the data of the regions we are merging.
Agreed but if we name the command like

{code}
merge table_name start_key end_key
{code}
then it means that clubbing multiple (2 or more) to a single region (provided 
data remains same)


 Provide capability to delete named region
 -

 Key: HBASE-4991
 URL: https://issues.apache.org/jira/browse/HBASE-4991
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch


 See discussion titled 'Able to control routing to Solr shards or not' on 
 lily-discuss
 User may want to quickly dispose of out of date records by deleting specific 
 regions. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5489) Add HTable accessor to get regions for a key range

[
https://issues.apache.org/jira/browse/HBASE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219692#comment-13219692
]

jirapos...@reviews.apache.org commented on HBASE-5489:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4117/
---

Review request for hbase.

Summary
---

getRegionsInRange() will retrieve the HRegionLocations for the regions
associated with the specified key range, using client-side cache if possible.

This addresses bug HBASE-5489.
https://issues.apache.org/jira/browse/HBASE-5489

Diffs
-

src/main/java/org/apache/hadoop/hbase/client/HTable.java 29b8004
src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java bdeaefe

Diff: https://reviews.apache.org/r/4117/diff

Testing
---

Ran the TestFromClientSide unit tests and passed repeatedly.

Ran test-patch.sh with the following results:

-1 overall.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -129 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version )
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

Thanks,

David

Add HTable accessor to get regions for a key range
--

Key: HBASE-5489
URL: https://issues.apache.org/jira/browse/HBASE-5489
Project: HBase
Issue Type: Improvement
Components: client
Reporter: David S. Wang
Assignee: David S. Wang
Priority: Minor

[jira] [Resolved] (HBASE-5498) S

2012-02-29 Thread Todd Lipcon (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon resolved HBASE-5498.


Resolution: Invalid

 S
 -

 Key: HBASE-5498
 URL: https://issues.apache.org/jira/browse/HBASE-5498
 Project: HBase
  Issue Type: Improvement
Reporter: Francis Liu



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5489) Add HTable accessor to get regions for a key range

2012-02-29 Thread David S. Wang (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

David S. Wang updated HBASE-5489:
-

Fix Version/s: 0.96.0
0.94.0
0.92.1
Status: Patch Available (was: In Progress)

The patch is in:

https://reviews.apache.org/r/4117/

I will submit it here for the Hadoop QA robot once it is approved.

Add HTable accessor to get regions for a key range
--

Key: HBASE-5489
URL: https://issues.apache.org/jira/browse/HBASE-5489
Project: HBase
Issue Type: Improvement
Components: client
Reporter: David S. Wang
Assignee: David S. Wang
Priority: Minor
Fix For: 0.92.1, 0.94.0, 0.96.0

[jira] [Created] (HBASE-5499) dev-support/test-patch.sh does not have execute perms

2012-02-29 Thread David S. Wang (Created) (JIRA)

dev-support/test-patch.sh does not have execute perms
-

 Key: HBASE-5499
 URL: https://issues.apache.org/jira/browse/HBASE-5499
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.94.0, 0.96.0
Reporter: David S. Wang
Priority: Trivial
 Fix For: 0.94.0, 0.96.0


When I checkout a tree from trunk, I notice that dev-support/test-patch.sh does 
not come with execute permissions enabled by default, while the rest of the 
scripts in dev-support do have +x set.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HBASE-5498) Secure Bulk Load

2012-02-29 Thread Francis Liu (Reopened) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Francis Liu reopened HBASE-5498:

sorry todd, submitted the jira before completing it.

Secure Bulk Load

Key: HBASE-5498
URL: https://issues.apache.org/jira/browse/HBASE-5498
Project: HBase
Issue Type: Improvement
Reporter: Francis Liu

Design doc:
https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load
Short summary:
Security as it stands does not cover the bulkLoadHFiles() feature. Users
calling this method will bypass ACLs. Also loading is made more cumbersome in
a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data
from user's directory to the hbase directory, which would require certain
write access privileges set.
Our solution is to create a coprocessor which makes use of AuthManager to
verify if a user has write access to the table. If so, launches a MR job as
the hbase user to do the importing (ie rewrite from text to hfiles). One
tricky part this job will have to do is impersonate the calling user when
reading the input files. We can do this by expecting the user to pass an hdfs
delegation token as part of the secureBulkLoad() coprocessor call and extend
an inputformat to make use of that token. The output is written to a
temporary directory accessible only by hbase and then bulkloadHFiles() is
called.

[jira] [Updated] (HBASE-5498) Secure Bulk Load

2012-02-29 Thread Francis Liu (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Francis Liu updated HBASE-5498:
---

Description:
Design doc:
https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load

Short summary:

Security as it stands does not cover the bulkLoadHFiles() feature. Users
calling this method will bypass ACLs. Also loading is made more cumbersome in a
secure setting because of hdfs privileges. bulkLoadHFiles() moves the data from
user's directory to the hbase directory, which would require certain write
access privileges set.

Our solution is to create a coprocessor which makes use of AuthManager to
verify if a user has write access to the table. If so, launches a MR job as the
hbase user to do the importing (ie rewrite from text to hfiles). One tricky
part this job will have to do is impersonate the calling user when reading the
input files. We can do this by expecting the user to pass an hdfs delegation
token as part of the secureBulkLoad() coprocessor call and extend an
inputformat to make use of that token. The output is written to a temporary
directory accessible only by hbase and then bulkloadHFiles() is called.

Summary: Secure Bulk Load (was: S)

Secure Bulk Load

Key: HBASE-5498
URL: https://issues.apache.org/jira/browse/HBASE-5498
Project: HBase
Issue Type: Improvement
Reporter: Francis Liu

[jira] [Commented] (HBASE-4991) Provide capability to delete named region

[
https://issues.apache.org/jira/browse/HBASE-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219706#comment-13219706
]

Mubarak Seyed commented on HBASE-4991:
--

bq. What do we have the regionserver do anything but close of the region? Why
do we delegate to it the deletion? Why not have it done by the master? Or a
client script? Have it remove the region from .META. and from the fs? And
bridge the hole in .META.? Isn't that less complicated?
Well, client's deleteRegion call is asynchronous so no fail-over if client has
to do the business.
Regarding master, it acts as a coordinator between client and RS, meaning it is
like move() region task (but split() goes from client to RS). Master does the
cleanup job of deleting the failed delete-region znodes if they exceeds the
configured timeout value (30 minutes)

{code}
this.deleteRegionTracker = new MasterDeleteRegionTracker(getZooKeeper(),
this,this, conf.getInt(hbase.delete.region.timeout, 180));
{code}
If we have to make client call to RS (as like compact or split) for
deleteRegion then how do we do clean-up? How about master-failover?

Provide capability to delete named region
-

Key: HBASE-4991
URL: https://issues.apache.org/jira/browse/HBASE-4991
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Mubarak Seyed
Fix For: 0.94.0

Attachments: HBASE-4991.trunk.v1.patch, HBASE-4991.trunk.v2.patch

See discussion titled 'Able to control routing to Solr shards or not' on
lily-discuss
User may want to quickly dispose of out of date records by deleting specific
regions.

[jira] [Updated] (HBASE-5491) Remove HBaseConfiguration.create() call from coprocessor.Exec class

2012-02-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

honghua zhu updated HBASE-5491:
---

Attachment: HBASE-5491-2.patch

 Remove HBaseConfiguration.create() call from coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491-2.patch, HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5498) Secure Bulk Load

2012-02-29 Thread Francis Liu (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219713#comment-13219713
]

Francis Liu commented on HBASE-5498:

The MR code will be part of hbase code just like importTSV. We can support
other file formats just not all of them.

Secure Bulk Load

Key: HBASE-5498
URL: https://issues.apache.org/jira/browse/HBASE-5498
Project: HBase
Issue Type: Improvement
Reporter: Francis Liu

[jira] [Commented] (HBASE-5489) Add HTable accessor to get regions for a key range

[
https://issues.apache.org/jira/browse/HBASE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219714#comment-13219714
]

jirapos...@reviews.apache.org commented on HBASE-5489:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4117/
---

(Updated 2012-03-01 01:51:42.601163)

Review request for hbase.

Changes
---

Added question to description about whether endKey should be inclusive or
exclusive.

Summary (updated)
---

getRegionsInRange() will retrieve the HRegionLocations for the regions
associated with the specified key range, using client-side cache if possible.

I have one question: right now the endKey specified to getRegionsInRange() is
treated as inclusive. I followed the behavior that I saw in
HRegionInfo.containsRange(). However, other HBase code such as Scan treats the
endKey as exclusive. So I am not clear as to which way we should go here. I
can easily change the patch if we want the endKey to be exclusive; please let
me know. Thanks in advance.

This addresses bug HBASE-5489.
https://issues.apache.org/jira/browse/HBASE-5489

Diffs
-

src/main/java/org/apache/hadoop/hbase/client/HTable.java 29b8004
src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java bdeaefe

Diff: https://reviews.apache.org/r/4117/diff

Testing
---

Ran the TestFromClientSide unit tests and passed repeatedly.

Ran test-patch.sh with the following results:

-1 overall.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -129 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version )
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

Thanks,

David

Add HTable accessor to get regions for a key range
--

[jira] [Commented] (HBASE-5489) Add HTable accessor to get regions for a key range

2012-02-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219723#comment-13219723
]

jirapos...@reviews.apache.org commented on HBASE-5489:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4117/#review5477
---

I'd go with exclusive. That way it is easy to have an inclusive (just add a
binary 0 at the end) and exclusive.
Patch looks good otherwise.

- Lars

On 2012-03-01 01:51:42, David Wang wrote:
bq.
bq. ---
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4117/
bq. ---
bq.
bq. (Updated 2012-03-01 01:51:42)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. ---
bq.
bq. getRegionsInRange() will retrieve the HRegionLocations for the regions
associated with the specified key range, using client-side cache if possible.
bq.
bq. I have one question: right now the endKey specified to getRegionsInRange()
is treated as inclusive. I followed the behavior that I saw in
HRegionInfo.containsRange(). However, other HBase code such as Scan treats the
endKey as exclusive. So I am not clear as to which way we should go here. I
can easily change the patch if we want the endKey to be exclusive; please let
me know. Thanks in advance.
bq.
bq.
bq. This addresses bug HBASE-5489.
bq. https://issues.apache.org/jira/browse/HBASE-5489
bq.
bq.
bq. Diffs
bq. -
bq.
bq.src/main/java/org/apache/hadoop/hbase/client/HTable.java 29b8004
bq.src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
bdeaefe
bq.
bq. Diff: https://reviews.apache.org/r/4117/diff
bq.
bq.
bq. Testing
bq. ---
bq.
bq. Ran the TestFromClientSide unit tests and passed repeatedly.
bq.
bq. Ran test-patch.sh with the following results:
bq.
bq. -1 overall.
bq.
bq. +1 @author. The patch does not contain any @author tags.
bq.
bq. +1 tests included. The patch appears to include 3 new or modified
tests.
bq.
bq. -1 javadoc. The javadoc tool appears to have generated -129 warning
messages.
bq.
bq. +1 javac. The applied patch does not increase the total number of
javac compiler warnings.
bq.
bq. +1 findbugs. The patch does not introduce any new Findbugs (version )
warnings.
bq.
bq. +1 release audit. The applied patch does not increase the total
number of release audit warnings.
bq.
bq.
bq. Thanks,
bq.
bq. David
bq.
bq.

Add HTable accessor to get regions for a key range
--

[jira] [Commented] (HBASE-5270) Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler

2012-02-29 Thread chunhui shen (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219727#comment-13219727
]

chunhui shen commented on HBASE-5270:
-

@stack
I think we could introduce initializing state first for HBASE-5454.

bq.Are there still holes? For example, say the .META. server crashes AFTER
we've verified it up in assignRootAndMeta but before we get to do a scan of
.META. to rebuild user regions list. Could .META. be assigned w/o log splitting
finishing?

Yes.it's another a hole, but it's easy to solve， we could stop the processing
of expired servers until master finished assign ROOT and META not initialized.

I have thought this issue for a long time, and I think preventing processing of
SSH is a clear and simple solution, otherwise we should consider many cases
where meta server died in different time during master initializing.

Handle potential data loss due to concurrent processing of processFaileOver
and ServerShutdownHandler
-

Key: HBASE-5270
URL: https://issues.apache.org/jira/browse/HBASE-5270
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Zhihong Yu
Assignee: chunhui shen
Fix For: 0.92.1, 0.94.0

[jira] [Commented] (HBASE-5454) Refuse operations from Admin befor master is initialized

2012-02-29 Thread chunhui shen (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219737#comment-13219737
 ] 

chunhui shen commented on HBASE-5454:
-

In patch v2, master will throw PleaseHoldException when master is initializing 
and client call admin operations.

 Refuse operations from Admin befor master is initialized
 

 Key: HBASE-5454
 URL: https://issues.apache.org/jira/browse/HBASE-5454
 Project: HBase
  Issue Type: Improvement
Reporter: chunhui shen
 Attachments: hbase-5454.patch, hbase-5454v2.patch


 In our testing environment,
 When master is initializing, we found conflict problems between 
 master#assignAllUserRegions and EnableTable event, causing assigning region 
 throw exception so that master abort itself.
 We think we'd better refuse operations from Admin, such as CreateTable, 
 EnableTable,etc, It could reduce error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5454) Refuse operations from Admin before master is initialized

2012-02-29 Thread chunhui shen (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunhui shen updated HBASE-5454:


Summary: Refuse operations from Admin before master is initialized  (was: 
Refuse operations from Admin befor master is initialized)

 Refuse operations from Admin before master is initialized
 -

 Key: HBASE-5454
 URL: https://issues.apache.org/jira/browse/HBASE-5454
 Project: HBase
  Issue Type: Improvement
Reporter: chunhui shen
 Attachments: hbase-5454.patch, hbase-5454v2.patch


 In our testing environment,
 When master is initializing, we found conflict problems between 
 master#assignAllUserRegions and EnableTable event, causing assigning region 
 throw exception so that master abort itself.
 We think we'd better refuse operations from Admin, such as CreateTable, 
 EnableTable,etc, It could reduce error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5489) Add HTable accessor to get regions for a key range

2012-02-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219750#comment-13219750
]

jirapos...@reviews.apache.org commented on HBASE-5489:
--

bq. On 2012-03-01 02:05:25, Lars Hofhansl wrote:
bq. I'd go with exclusive. That way it is easy to have an inclusive (just
add a binary 0 at the end) and exclusive.
bq. Patch looks good otherwise.

Thanks! I'll redo the patch and retest. New patch should be up soon.

- David

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4117/#review5477
---

Add HTable accessor to get regions for a key range
--

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA

[jira] [Commented] (HBASE-5491) Remove HBaseConfiguration.create() call from coprocessor.Exec class

2012-02-29 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219755#comment-13219755
 ] 

Zhihong Yu commented on HBASE-5491:
---

Patch v2 looks good.

 Remove HBaseConfiguration.create() call from coprocessor.Exec class
 ---

 Key: HBASE-5491
 URL: https://issues.apache.org/jira/browse/HBASE-5491
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
 Environment: all
Reporter: honghua zhu
 Fix For: 0.92.1

 Attachments: HBASE-5491-2.patch, HBASE-5491.patch


 Exec class has a field: private Configuration conf = 
 HBaseConfiguration.create()
 Client side generates an Exec instance of the class, each initiated 
 Statistics request by ExecRPCInvoker
 Is so HBaseConfiguration.create for each request needs to call
 When the server side deserialize the Exec Called once 
 HBaseConfiguration.create in,
 HBaseConfiguration.create is a time consuming operation.
 private Configuration conf = HBaseConfiguration.create();
 This code is only useful for testing code 
 (org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testExecDeserialization),
 other places with the Exec class, pass a Configuration come,
 so no need to conf field a default value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5451) Switch RPC call envelope/headers to PBs


[ 
https://issues.apache.org/jira/browse/HBASE-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13219774#comment-13219774
 ] 

jirapos...@reviews.apache.org commented on HBASE-5451:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4096/
---

(Updated 2012-03-01 03:40:14.496251)


Review request for .


Changes
---

Updated with more doc.


Summary
---

Switch RPC call envelope/headers to PBs


This addresses bug HBASE-5451.
https://issues.apache.org/jira/browse/HBASE-5451


Diffs (updated)
-

  http://svn.apache.org/repos/asf/hbase/trunk/pom.xml 1294899 
  
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/DataOutputOutputStream.java
 1294899 
  
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
 1294899 
  
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java
 1294899 
  
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/User.java
 1294899 
  
http://svn.apache.org/repos/asf/hbase/trunk/src/main/proto/RPCMessageProto.proto
 PRE-CREATION 

Diff: https://reviews.apache.org/r/4096/diff


Testing
---


Thanks,

Devaraj



 Switch RPC call envelope/headers to PBs
 ---

 Key: HBASE-5451
 URL: https://issues.apache.org/jira/browse/HBASE-5451
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Todd Lipcon
Assignee: Devaraj Das
 Attachments: rpc-proto.patch.1_2




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5454) Refuse operations from Admin before master is initialized