[jira] [Commented] (HBASE-5372) Table mutation operations should check table level rights, not global rights

[Andrew Purtell (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205282#comment-13205282
 ] 

Andrew Purtell commented on HBASE-5372:
---

bq. However, for this maybe we have to revisit table ownership rights. 
Currently, the table owner has every right on the table, and this is not 
managed through the normal grant/revoke operations, but on the table metadata. 
We may want to remove table ownership, but introduce default table creation 
rights, which means that when a user creates a table, she automatically get 
those rights allocated. But another user can grant extra rights, or revoke 
them. 

Sure, makes sense. We opted for simplicity in the initial implementation.

> Table mutation operations should check table level rights, not global rights 
> -
>
> Key: HBASE-5372
> URL: https://issues.apache.org/jira/browse/HBASE-5372
> Project: HBase
>  Issue Type: Sub-task
>  Components: security
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> getUserPermissions(tableName)/grant/revoke and drop/modify table operations 
> should not check for global CREATE/ADMIN rights, but table CREATE/ADMIN 
> rights. The reasoning is that if a user is able to admin or read from a 
> table, she should be able to read the table's permissions. We can choose 
> whether we want only READ or ADMIN permissions for getUserPermission(). Since 
> we check for global permissions first for table permissions, configuring 
> table access using global permissions will continue to work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205281#comment-13205281
 ] 

jirapos...@reviews.apache.org commented on HBASE-5371:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3829/#review5003
---



security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java


This seems a reasonable approach.



security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java


Perhaps add a couple of additional cases to check?



security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java


Likewise


- Andrew


On 2012-02-09 23:12:00, enis wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3829/
bq.  ---
bq.  
bq.  (Updated 2012-02-09 23:12:00)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  We need to introduce something like 
AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
that clients can check access rights before carrying out the operations. We 
need this kind of operation for HCATALOG-245, which introduces authorization 
providers for hbase over hcat. We cannot use getUserPermissions() since it 
requires ADMIN permissions on the global/table level.
bq.  
bq.  
bq.  This addresses bug HBASE-5371.
bq.  https://issues.apache.org/jira/browse/HBASE-5371
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
 5091b7d 
bq.
security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
 5fa2edb 
bq.
security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 f864373 
bq.  
bq.  Diff: https://reviews.apache.org/r/3829/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  enis
bq.  
bq.



> Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) 
> API
> 
>
> Key: HBASE-5371
> URL: https://issues.apache.org/jira/browse/HBASE-5371
> Project: HBase
>  Issue Type: Sub-task
>  Components: security
>Affects Versions: 0.94.0, 0.92.1
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> We need to introduce something like 
> AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
> that clients can check access rights before carrying out the operations. We 
> need this kind of operation for HCATALOG-245, which introduces authorization 
> providers for hbase over hcat. We cannot use getUserPermissions() since it 
> requires ADMIN permissions on the global/table level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[David S. Wang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David S. Wang updated HBASE-5209:
-

Fix Version/s: 0.90.7

> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: HBASE-5209-v0.diff, HBASE-5209-v1.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[David S. Wang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David S. Wang updated HBASE-5209:
-

Status: Patch Available  (was: Open)

Ran this one past test-patch.

> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.92.0, 0.90.5, 0.94.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5209-v0.diff, HBASE-5209-v1.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[David S. Wang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David S. Wang updated HBASE-5209:
-

Status: Open  (was: Patch Available)

Need to resubmit new patch with correct formatting.

> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.92.0, 0.90.5, 0.94.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5209-v0.diff, HBASE-5209-v1.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[David S. Wang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David S. Wang updated HBASE-5209:
-

Attachment: HBASE-5209-v1.diff

Ran this one past test-patch.

> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5209-v0.diff, HBASE-5209-v1.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[David S. Wang (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205270#comment-13205270
 ] 

David S. Wang commented on HBASE-5209:
--

I also ran "hbase hbck -details" successfully in order to exercise the changes 
in HBaseFsck.java.

> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5209-v0.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205269#comment-13205269
 ] 

Hadoop QA commented on HBASE-5209:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514082/HBASE-5209-v0.diff
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/938//console

This message is automatically generated.

> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5209-v0.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[David S. Wang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David S. Wang updated HBASE-5209:
-

Attachment: HBASE-5209-v0.diff

> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5209-v0.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

[David S. Wang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David S. Wang updated HBASE-5209:
-

Fix Version/s: 0.92.1
   0.94.0
   Status: Patch Available  (was: Open)

I tested this by:

* Running "mvn -P localTests -Dtest=TestMasterFailover test" successfully
* Starting a master and a couple of backup masters, checking zk_dump, killing 
the master, checking zk_dump, restarting the master, checking zk_dump.  All 
checks displayed expected values in the master and backupmaster ZNodes.


> HConnection/HMasterInterface should allow for way to get hostname of 
> currently active master in multi-master HBase setup
> 
>
> Key: HBASE-5209
> URL: https://issues.apache.org/jira/browse/HBASE-5209
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 0.92.0, 0.90.5, 0.94.0
>Reporter: Aditya Acharya
>Assignee: David S. Wang
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5209-v0.diff
>
>
> I have a multi-master HBase set up, and I'm trying to programmatically 
> determine which of the masters is currently active. But the API does not 
> allow me to do this. There is a getMaster() method in the HConnection class, 
> but it returns an HMasterInterface, whose methods do not allow me to find out 
> which master won the last race. The API should have a 
> getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans

[Hitesh Shah (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205260#comment-13205260
 ] 

Hitesh Shah commented on HBASE-5325:


bq. No, metrics2 is not in CDH, since it's neither compatible with earlier 
versions of metrics1 nor is it compatible with the metrics2 in 0.23+.
Will make the change to use metrics.MBeanUtil#registerMBean in that case. I am 
assuming that will meet all branches' needs?


> Expose basic information about the master-status through jmx beans 
> ---
>
> Key: HBASE-5325
> URL: https://issues.apache.org/jira/browse/HBASE-5325
> Project: HBase
>  Issue Type: Improvement
>Reporter: Hitesh Shah
>Assignee: Hitesh Shah
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch
>
>
> Similar to the Namenode and Jobtracker, it would be good if the hbase master 
> could expose some information through mbeans.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5075) regionserver crashed and failover

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205258#comment-13205258
 ] 

Zhihong Yu commented on HBASE-5075:
---

One solution is to expire region server znode more quickly.

> regionserver crashed and failover
> -
>
> Key: HBASE-5075
> URL: https://issues.apache.org/jira/browse/HBASE-5075
> Project: HBase
>  Issue Type: Improvement
>  Components: monitoring, regionserver, replication, zookeeper
>Affects Versions: 0.92.1
>Reporter: 代志远
> Fix For: 0.90.5
>
>
> regionserver crashed,it is too long time to notify hmaster.when hmaster know 
> regionserver's shutdown,it is long time to fetch the hlog's lease.
> hbase is a online db, availability is very important.
> i have a idea to improve availability, monitor node to check regionserver's 
> pid.if this pid not exsits,i think the rs down,i will delete the znode,and 
> force close the hlog file.
> so the period maybe 100ms.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205255#comment-13205255
 ] 

Hudson commented on HBASE-5358:
---

Integrated in HBase-TRUNK-security #104 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/104/])
HBASE-5358 HBaseObjectWritable should be able to serialize/deserialize 
generic arrays

tedyu : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java


> HBaseObjectWritable should be able to serialize/deserialize generic arrays
> --
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Affects Versions: 0.94.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.94.0
>
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205251#comment-13205251
 ] 

Todd Lipcon commented on HBASE-5325:


No, metrics2 is not in CDH, since it's neither compatible with earlier versions 
of metrics1 nor is it compatible with the metrics2 in 0.23+.

> Expose basic information about the master-status through jmx beans 
> ---
>
> Key: HBASE-5325
> URL: https://issues.apache.org/jira/browse/HBASE-5325
> Project: HBase
>  Issue Type: Improvement
>Reporter: Hitesh Shah
>Assignee: Hitesh Shah
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch
>
>
> Similar to the Namenode and Jobtracker, it would be good if the hbase master 
> could expose some information through mbeans.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5075) regionserver crashed and failover

[Updated]

 [ 
https://issues.apache.org/jira/browse/HBASE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

代志远 updated HBASE-5075:
---

Fix Version/s: 0.90.5

> regionserver crashed and failover
> -
>
> Key: HBASE-5075
> URL: https://issues.apache.org/jira/browse/HBASE-5075
> Project: HBase
>  Issue Type: Improvement
>  Components: monitoring, regionserver, replication, zookeeper
>Affects Versions: 0.92.1
>Reporter: 代志远
> Fix For: 0.90.5
>
>
> regionserver crashed,it is too long time to notify hmaster.when hmaster know 
> regionserver's shutdown,it is long time to fetch the hlog's lease.
> hbase is a online db, availability is very important.
> i have a idea to improve availability, monitor node to check regionserver's 
> pid.if this pid not exsits,i think the rs down,i will delete the znode,and 
> force close the hlog file.
> so the period maybe 100ms.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5376) Add more logging to triage HBASE-5312: Closed parent region present in Hlog.lastSeqWritten

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205250#comment-13205250
 ] 

Todd Lipcon commented on HBASE-5376:


I bet we could also write a stress test - sounds like some race where we don't 
block off updates at the right spot when closing a region. Maybe a test which 
continually opens/closes a region while making edits, and after every close, 
verify it's not in the map?

> Add more logging to triage HBASE-5312: Closed parent region present in 
> Hlog.lastSeqWritten
> --
>
> Key: HBASE-5376
> URL: https://issues.apache.org/jira/browse/HBASE-5376
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jimmy Xiang
>Priority: Minor
> Fix For: 0.90.7
>
>
> It is hard to find out what exactly caused HBASE-5312.  Some logging will be 
> helpful to shine some lights.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5377) Fix licenses on the 0.90 branch.

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205239#comment-13205239
 ] 

Hadoop QA commented on HBASE-5377:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514077/hbase-5377.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 18 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/937//console

This message is automatically generated.

> Fix licenses on the 0.90 branch.
> 
>
> Key: HBASE-5377
> URL: https://issues.apache.org/jira/browse/HBASE-5377
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5377.patch
>
>
> There are a handful of empty files and several files missing apache licenses 
> on the 0.90 branch.  This patch will fixes all of them and in conjunction 
> with HBASE-5363 will allow it to pass RAT tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5377) Fix licenses on the 0.90 branch.

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205240#comment-13205240
 ] 

Jonathan Hsieh commented on HBASE-5377:
---

server info web pages look good. 

'mvn site' to build docs seem to have a problem, but this is present before and 
after.

{code}
...
[INFO] --- maven-site-plugin:2.0.1:site (default-site) @ hbase ---
[INFO] Unable to load parent project from a relative path: 1 problem was 
encountered while building the effective model
[FATAL] Non-readable POM /home/jon/proj/pom.xml: /home/jon/proj/pom.xml (No 
such file or directory) @ 
 for project  at /home/jon/proj/pom.xml for project  at /home/jon/proj/pom.xml
[INFO] Parent project loaded from repository.
[INFO] Unable to load parent project from a relative path: 1 problem was 
encountered while building the effective model
[FATAL] Non-readable POM /home/jon/proj/pom.xml: /home/jon/proj/pom.xml (No 
such file or directory) @ 
 for project  at /home/jon/proj/pom.xml for project  at /home/jon/proj/pom.xml
[INFO] Parent project loaded from repository.
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
...
{code}

> Fix licenses on the 0.90 branch.
> 
>
> Key: HBASE-5377
> URL: https://issues.apache.org/jira/browse/HBASE-5377
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5377.patch
>
>
> There are a handful of empty files and several files missing apache licenses 
> on the 0.90 branch.  This patch will fixes all of them and in conjunction 
> with HBASE-5363 will allow it to pass RAT tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans

[Hitesh Shah (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205234#comment-13205234
 ] 

Hitesh Shah commented on HBASE-5325:


@Stack, thanks for the comments. Will go over them and submit a modified patch 
soon.

> Expose basic information about the master-status through jmx beans 
> ---
>
> Key: HBASE-5325
> URL: https://issues.apache.org/jira/browse/HBASE-5325
> Project: HBase
>  Issue Type: Improvement
>Reporter: Hitesh Shah
>Assignee: Hitesh Shah
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch
>
>
> Similar to the Namenode and Jobtracker, it would be good if the hbase master 
> could expose some information through mbeans.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5377) Fix licenses on the 0.90 branch.

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5377:
--

Status: Patch Available  (was: Open)

> Fix licenses on the 0.90 branch.
> 
>
> Key: HBASE-5377
> URL: https://issues.apache.org/jira/browse/HBASE-5377
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Jonathan Hsieh
> Attachments: hbase-5377.patch
>
>
> There are a handful of empty files and several files missing apache licenses 
> on the 0.90 branch.  This patch will fixes all of them and in conjunction 
> with HBASE-5363 will allow it to pass RAT tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5377) Fix licenses on the 0.90 branch.

[Jonathan Hsieh (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh reassigned HBASE-5377:
-

Assignee: Jonathan Hsieh

> Fix licenses on the 0.90 branch.
> 
>
> Key: HBASE-5377
> URL: https://issues.apache.org/jira/browse/HBASE-5377
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5377.patch
>
>
> There are a handful of empty files and several files missing apache licenses 
> on the 0.90 branch.  This patch will fixes all of them and in conjunction 
> with HBASE-5363 will allow it to pass RAT tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5377) Fix licenses on the 0.90 branch.

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5377:
--

Attachment: hbase-5377.patch

> Fix licenses on the 0.90 branch.
> 
>
> Key: HBASE-5377
> URL: https://issues.apache.org/jira/browse/HBASE-5377
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Jonathan Hsieh
> Attachments: hbase-5377.patch
>
>
> There are a handful of empty files and several files missing apache licenses 
> on the 0.90 branch.  This patch will fixes all of them and in conjunction 
> with HBASE-5363 will allow it to pass RAT tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5377) Fix licenses on the 0.90 branch.

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205225#comment-13205225
 ] 

Jonathan Hsieh commented on HBASE-5377:
---

This is partially a backport of HBASE-4647 but also fixes other files that are 
only relevant on the 0.90 branch.

> Fix licenses on the 0.90 branch.
> 
>
> Key: HBASE-5377
> URL: https://issues.apache.org/jira/browse/HBASE-5377
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Jonathan Hsieh
>
> There are a handful of empty files and several files missing apache licenses 
> on the 0.90 branch.  This patch will fixes all of them and in conjunction 
> with HBASE-5363 will allow it to pass RAT tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5377) Fix licenses on the 0.90 branch.

[Jonathan Hsieh (Created) (JIRA)]

Fix licenses on the 0.90 branch.


 Key: HBASE-5377
 URL: https://issues.apache.org/jira/browse/HBASE-5377
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jonathan Hsieh


There are a handful of empty files and several files missing apache licenses on 
the 0.90 branch.  This patch will fixes all of them and in conjunction with 
HBASE-5363 will allow it to pass RAT tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5376) Add more logging to triage HBASE-5312: Closed parent region present in Hlog.lastSeqWritten

[Jimmy Xiang (Created) (JIRA)]

Add more logging to triage HBASE-5312: Closed parent region present in 
Hlog.lastSeqWritten
--

 Key: HBASE-5376
 URL: https://issues.apache.org/jira/browse/HBASE-5376
 Project: HBase
  Issue Type: Sub-task
Reporter: Jimmy Xiang
Priority: Minor


It is hard to find out what exactly caused HBASE-5312.  Some logging will be 
helpful to shine some lights.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5375) Ensure that compactions use already cached blocks but do not cache new data blocks

[Mikhail Bautin (Created) (JIRA)]

Ensure that compactions use already cached blocks but do not cache new data 
blocks
--

 Key: HBASE-5375
 URL: https://issues.apache.org/jira/browse/HBASE-5375
 Project: HBase
  Issue Type: Test
Reporter: Mikhail Bautin


Create a unit test to verify that compactions reuse existing cached blocks but 
do not thrash the cache with newly read blocks. Also need to verify that we 
only read every data block once, e.g. that we don't re-read the block on every 
next() operation. HBASE-1597 did not seem to include a unit test, so we need to 
add a test now. This and HBASE-4683 (the unit test that was not checked in) are 
the remaining missing pieces before we can close HBASE-3976.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4683) Always cache index and bloom blocks

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-4683:
---

Attachment: D1695.1.patch

mbautin requested code review of "[jira] [HBASE-4683] Test that we always cache 
index and bloom blocks".
Reviewers: JIRA, jdcryans, lhofhansl, Liyin

  This was reviewed D807 but was not checked in. Submitting unit test as a 
separate patch, and extending the scope of the test to also handle the case 
when block cache is enabled for the column family.

TEST PLAN
  Run unit tests

REVISION DETAIL
  https://reviews.facebook.net/D1695

AFFECTED FILES
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
  
src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/3621/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> Always cache index and bloom blocks
> ---
>
> Key: HBASE-4683
> URL: https://issues.apache.org/jira/browse/HBASE-4683
> Project: HBase
>  Issue Type: New Feature
>Reporter: Lars Hofhansl
>Assignee: Mikhail Bautin
>Priority: Minor
> Fix For: 0.94.0, 0.92.0
>
> Attachments: 0001-Cache-important-block-types.patch, 4683-v2.txt, 
> 4683.txt, D1695.1.patch, D807.1.patch, D807.2.patch, D807.3.patch, 
> HBASE-4683-0.92-v2.patch, HBASE-4683-v3.patch
>
>
> This would add a new boolean config option: hfile.block.cache.datablocks
> Default would be true.
> Setting this to false allows HBase in a mode where only index blocks are 
> cached, which is useful for analytical scenarios where a useful working set 
> of the data cannot be expected to fit into the (aggregate) cache.
> This is the equivalent of setting cacheBlocks to false on all scans 
> (including scans on behalf of gets).
> I would like to get a general feeling about what folks think about this.
> The change itself would be simple.
> Update (Mikhail): we probably don't need a new conf option. Instead, we will 
> make index blocks cached by default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205215#comment-13205215
 ] 

stack commented on HBASE-5325:
--

I suppose you can't name the bean for the ServerName because you need to be 
able to locate the bean -- i.e. there'll be N instances of RegionServer beans 
and you'd then figure which belonged to which by looking at the ServerName 
attribute (I remember this is how it worked now).

Is this in branch-0.20-append branch:

+import org.apache.hadoop.metrics2.util.MBeans;

If not, this breaks our ability to run on that branch (Up to this presumed we 
could run there).  Its in 1.0.0 hadoop?  (would need to check its in CDH..)

Do classes have to have an HBase (or Hbase) prefix? Seems redundant (and we 
should be consistent).

Can we have a better name than HBaseRegionServerInfo.  It says nothing.  If it 
was RegionServerMBean or MXBean, it'd say more about what this class is about.

You can all it 'instance'.  'theInstance' is too much?

And again, publishing master, ensemble, and startcode seems like a bunch of 
info you'd never act on.  Master maybe -- you'd know which master it was 
talking too -- and perhaps ensemble because then you know who its registered 
with (though having both seems unnessary... the ensemble with its rootdir will 
tell you which cluster we belong too... perhaps you should get the cluster id 
out here?) but startcode is no good to anyone really.  Should be ServerName 
coming out here.  Thats how we uniquely identify regionservers in fs, when they 
report into the master and up in ensemble.  Might as well continue the 
identifier here.

Does this class need to take a RegionServer implementation?  Can it take a 
o.a.h.h.Server and/or a o.a.h.h.regionserver.RegionServerServices publishing 
jmx attributes?  These are Interfaces.  Might make this all easier to write 
tests on.


HbaseRegionServerMXBean should be in the regionserver/metrics package then you 
could call it MXBean or ServerMXBean.


HBaseMasterMXBean should be min master/metrics, should be HBaseMasterMXBean.  
The RegionServerInfo in it should be showing more than this small set of 
metrics if you are going to the bother of putting this stuff out on jmx (we do 
requests per region now -- should there be one of these classes per region?)  

I think that you don't need startcode if getRegionServer is returning the 
ServerName as a String.

Don't RegionState up in RegionsInTransiation have a ServerName associated too?  
YOu're not publishing this?

Again, ain't these bad names for beans up in jmx?

+mxBean = MBeans.register("HBase", "MasterInfo",
+mxBeanInfo);

shouldn't it be org.apache.hbase... or hbase if thats the parent for all of the 
hbase beans (Ain't there convention on bean naming -- IIRC).  MasterInfo says 
nothing.  Could it be just Master.  Then you do getServerName or what ever it 
is that returns ServerName to distingush this master from the backup Master?

Sorry for so many comments for such a small patch.  I just feel that this stuff 
can be really useful if its done right.  Else its just more stuff for us to 
maintain.  Thanks for doing this stuff lads.

> Expose basic information about the master-status through jmx beans 
> ---
>
> Key: HBASE-5325
> URL: https://issues.apache.org/jira/browse/HBASE-5325
> Project: HBase
>  Issue Type: Improvement
>Reporter: Hitesh Shah
>Assignee: Hitesh Shah
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch
>
>
> Similar to the Namenode and Jobtracker, it would be good if the hbase master 
> could expose some information through mbeans.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5312) Closed parent region present in Hlog.lastSeqWritten

[Jimmy Xiang (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205214#comment-13205214
 ] 

Jimmy Xiang commented on HBASE-5312:


Have anyone seen this issue on 0.92 release?  Could we add some logging so that 
we will have some clue when it happens again?

> Closed parent region present in Hlog.lastSeqWritten
> ---
>
> Key: HBASE-5312
> URL: https://issues.apache.org/jira/browse/HBASE-5312
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
>
> This is in reference to the mail sent in the dev mailing list
> "Closed parent region present in Hlog.lastSeqWritten".
> The sceanrio described is
> We had a region that was split into two daughters.  When the hlog roll tried 
> to flush the region there was an entry in the HLog.lastSeqWritten that was 
> not flushed or removed from the lastSeqWritten during the parent close.
> Because this flush was not happening subsequent flushes were getting blocked
> {code}
>  05:06:44,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
>  hlogs: logs=122, maxlogs=32; forcing flush of 1 regions(s):
>  2acaf8e3acfd2e8a5825a1f6f0aca4a8
>  05:06:44,422 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
>  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
> requester=null
>  05:10:48,666 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
>  hlogs: logs=123, maxlogs=32; forcing flush of 1 regions(s):
>  2acaf8e3acfd2e8a5825a1f6f0aca4a8
>  05:10:48,666 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
>  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
> requester=null
>  05:14:46,075 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
>  hlogs: logs=124, maxlogs=32; forcing flush of 1 regions(s):
>  2acaf8e3acfd2e8a5825a1f6f0aca4a8
>  05:14:46,075 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
>  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
> requester=null
>  05:15:41,584 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
>  hlogs: logs=125, maxlogs=32; forcing flush of 1 regions(s):
>  2acaf8e3acfd2e8a5825a1f6f0aca4a8
>  05:15:41,584 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
>  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
> {code}
> Lets see what happened for the region 2acaf8e3acfd2e8a5825a1f6f0aca4a8
> {code}
> 2012-01-06 00:30:55,214 INFO org.apache.hadoop.hbase.regionserver.Store: 
> Renaming flushed file at 
> hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/.tmp/1755862026714756815
>  to 
> hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123
> 2012-01-06 00:30:58,946 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> Instantiated 
> Htable_UFDR_016,049790700093168-0456520,1325809837958.0ebe5bd7fcbc09ee074d5600b9d4e062.
> 2012-01-06 00:30:59,614 INFO org.apache.hadoop.hbase.regionserver.Store: 
> Added 
> hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123,
>  entries=7537, sequenceid=20312223, memsize=4.2m, filesize=2.9m
> 2012-01-06 00:30:59,787 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> Finished snapshotting, commencing flushing stores
> 2012-01-06 00:30:59,787 INFO org.apache.hadoop.hbase.regionserver.HRegion: 
> Finished memstore flush of ~133.5m for region 
> Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in 
> 21816ms, sequenceid=20312223, compaction requested=true
> 2012-01-06 00:30:59,787 DEBUG 
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested 
> for Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. 
> because regionserver20020.cacheFlusher; priority=0, compaction queue size=5840
> {code}
> A user triggered split has been issued to this region which can be seen in 
> the above logs.
> The flushing of this region has resulted in a seq id 20312223.
> The region has been splitted and the parent region has been closed
> {code}
> 00:31:12,607 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: 
> Starting split of region 
> Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
> 00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing 
> Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.: 
> disabling compactions & flushes
> 00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates 
> disabled for region 
> Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
> 00:31:13,718 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed 
> Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5

[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205210#comment-13205210
 ] 

Jonathan Hsieh commented on HBASE-5327:
---

Jimmy and I were chatting about the funny control flow and found it strange 
that {{new Path(invalidRootDir)}} didn't through IllegalArgumentException, 
while {{ new Path (new Path(invalidRootDir), ".olddirs")}} threw an 
IllegalArguentException.  (I was expecting the mkdirs to IAE, not exists).  

This explains why the slightly odd control flow is necessary and makes me much 
happier with v2 patch.

Maybe someone with more HDFS background can chime in about why this seeming 
inconsistency exists?




> Print a message when an invalid hbase.rootdir is passed
> ---
>
> Key: HBASE-5327
> URL: https://issues.apache.org/jira/browse/HBASE-5327
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: Jean-Daniel Cryans
>Assignee: Jimmy Xiang
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: hbase-5327.txt, hbase-5327_v2.txt
>
>
> As seen on the mailing list: 
> http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
> If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a 
> path to .oldlogs:
> {noformat}
> 2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster: 
> Unhandled exception. Starting shutdown.
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
> at org.apache.hadoop.fs.Path.initialize(Path.java:148)
> at org.apache.hadoop.fs.Path.(Path.java:71)
> at org.apache.hadoop.fs.Path.(Path.java:50)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:112)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> hdfs://sv4r11s38:9100.oldlogs
> at java.net.URI.checkPath(URI.java:1787)
> at java.net.URI.(URI.java:735)
> at org.apache.hadoop.fs.Path.initialize(Path.java:145)
> ... 6 more
> {noformat}
> It could also crash anywhere else, this just happens to be the first place we 
> use hbase.rootdir. We need to verify that it's an actual folder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5368) Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible in HBase installs

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205204#comment-13205204
 ] 

Hadoop QA commented on HBASE-5368:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514067/5368.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 6 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -136 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 157 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/936//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/936//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/936//console

This message is automatically generated.

> Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible 
> in HBase installs
> -
>
> Key: HBASE-5368
> URL: https://issues.apache.org/jira/browse/HBASE-5368
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 5368.txt
>
>
> Very simple change to make PrefixSplitKeyPolicy accessible in HBase installs 
> (user still needs to setup the table(s) accordingly).
> Right now it is in src/test/org.apache.hadoop.hbase.regionserver, I propose 
> moving it to src/org.apache.hadoop.hbase.regionserver (alongside 
> ConstantSizeRegionSplitPolicy), and maybe renaming it too.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4683) Always cache index and bloom blocks

[Mikhail Bautin (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205200#comment-13205200
 ] 

Mikhail Bautin commented on HBASE-4683:
---

@jdcryans: it looks like TestForceCacheImportantBlocks.java was not included 
into your trunk commit of this JIRA. I will verify that the test still works 
and post an addendum patch.

> Always cache index and bloom blocks
> ---
>
> Key: HBASE-4683
> URL: https://issues.apache.org/jira/browse/HBASE-4683
> Project: HBase
>  Issue Type: New Feature
>Reporter: Lars Hofhansl
>Assignee: Mikhail Bautin
>Priority: Minor
> Fix For: 0.94.0, 0.92.0
>
> Attachments: 0001-Cache-important-block-types.patch, 4683-v2.txt, 
> 4683.txt, D807.1.patch, D807.2.patch, D807.3.patch, HBASE-4683-0.92-v2.patch, 
> HBASE-4683-v3.patch
>
>
> This would add a new boolean config option: hfile.block.cache.datablocks
> Default would be true.
> Setting this to false allows HBase in a mode where only index blocks are 
> cached, which is useful for analytical scenarios where a useful working set 
> of the data cannot be expected to fit into the (aggregate) cache.
> This is the equivalent of setting cacheBlocks to false on all scans 
> (including scans on behalf of gets).
> I would like to get a general feeling about what folks think about this.
> The change itself would be simple.
> Update (Mikhail): we probably don't need a new conf option. Instead, we will 
> make index blocks cached by default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205196#comment-13205196
 ] 

Zhihong Yu commented on HBASE-5358:
---

Integrated to TRUNK.

Thanks for the patch Enis.

Thanks for the review Michael.

> HBaseObjectWritable should be able to serialize/deserialize generic arrays
> --
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Affects Versions: 0.94.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.94.0
>
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays

[Zhihong Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5358:
--

Fix Version/s: 0.94.0
 Hadoop Flags: Reviewed
  Summary: HBaseObjectWritable should be able to serialize/deserialize 
generic arrays  (was: HBaseObjectWritable should be able to serialize generic 
arrays not defined previously)

> HBaseObjectWritable should be able to serialize/deserialize generic arrays
> --
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Affects Versions: 0.94.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.94.0
>
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5368) Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible in HBase installs

[Zhihong Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5368:
--

Status: Patch Available  (was: Open)

> Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible 
> in HBase installs
> -
>
> Key: HBASE-5368
> URL: https://issues.apache.org/jira/browse/HBASE-5368
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 5368.txt
>
>
> Very simple change to make PrefixSplitKeyPolicy accessible in HBase installs 
> (user still needs to setup the table(s) accordingly).
> Right now it is in src/test/org.apache.hadoop.hbase.regionserver, I propose 
> moving it to src/org.apache.hadoop.hbase.regionserver (alongside 
> ConstantSizeRegionSplitPolicy), and maybe renaming it too.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5230) Ensure compactions do not cache-on-write data blocks

[Mikhail Bautin (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Bautin updated HBASE-5230:
--

Release Note:   (was: Committed into both trunk and 89-fb)

> Ensure compactions do not cache-on-write data blocks
> 
>
> Key: HBASE-5230
> URL: https://issues.apache.org/jira/browse/HBASE-5230
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
>Priority: Minor
> Attachments: D1353.1.patch, D1353.2.patch, D1353.3.patch, 
> D1353.4.patch, 
> Don-t-cache-data-blocks-on-compaction-2012-01-21_00_53_54.patch, 
> Don-t-cache-data-blocks-on-compaction-2012-01-23_10_23_45.patch, 
> Don-t-cache-data-blocks-on-compaction-2012-01-23_15_27_23.patch
>
>
> Create a unit test for HBASE-3976 (making sure we don't cache data blocks on 
> write during compactions even if cache-on-write is enabled generally 
> enabled). This is because we have very different implementations of 
> HBASE-3976 without HBASE-4422 CacheConfig (on top of 89-fb, created by Liyin) 
> and with CacheConfig (presumably it's there but not sure if it even works, 
> since the patch in HBASE-3976 may not have been committed). We need to create 
> a unit test to verify that we don't cache data blocks on write during 
> compactions, and resolve HBASE-3976 so that this new unit test does not fail.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5230) Ensure compactions do not cache-on-write data blocks

[Mikhail Bautin (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Bautin updated HBASE-5230:
--

  Resolution: Fixed
Release Note: Committed into both trunk and 89-fb
  Status: Resolved  (was: Patch Available)

> Ensure compactions do not cache-on-write data blocks
> 
>
> Key: HBASE-5230
> URL: https://issues.apache.org/jira/browse/HBASE-5230
> Project: HBase
>  Issue Type: Improvement
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
>Priority: Minor
> Attachments: D1353.1.patch, D1353.2.patch, D1353.3.patch, 
> D1353.4.patch, 
> Don-t-cache-data-blocks-on-compaction-2012-01-21_00_53_54.patch, 
> Don-t-cache-data-blocks-on-compaction-2012-01-23_10_23_45.patch, 
> Don-t-cache-data-blocks-on-compaction-2012-01-23_15_27_23.patch
>
>
> Create a unit test for HBASE-3976 (making sure we don't cache data blocks on 
> write during compactions even if cache-on-write is enabled generally 
> enabled). This is because we have very different implementations of 
> HBASE-3976 without HBASE-4422 CacheConfig (on top of 89-fb, created by Liyin) 
> and with CacheConfig (presumably it's there but not sure if it even works, 
> since the patch in HBASE-3976 may not have been committed). We need to create 
> a unit test to verify that we don't cache data blocks on write during 
> compactions, and resolve HBASE-3976 so that this new unit test does not fail.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-4516) HFile-level load tester with compaction and random-read workloads

[Mikhail Bautin (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Bautin resolved HBASE-4516.
---

Resolution: Fixed

Resolved as part of HBASE-4218.

> HFile-level load tester with compaction and random-read workloads
> -
>
> Key: HBASE-4516
> URL: https://issues.apache.org/jira/browse/HBASE-4516
> Project: HBase
>  Issue Type: Test
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
>Priority: Minor
> Fix For: 0.94.0
>
>
> This is a load testing tool for HFile implementations, which supports two 
> workloads:
> - Compactions (merge the input HFiles). A special case of this is only one 
> input, which allows to do HFile format conversions.
> - Random reads. Launches the specified number of threads that do seeks and 
> short scans on randomly generated keys.
> The original purpose of this tool was to ensure that HFile format v2 did not 
> introduce performance regressions.
> Keys for the read workload are generated randomly between the first and the 
> last key of the HFile. At each position, instead of precisely calculating the 
> correct probability for every byte value b, we select a uniformly random byte 
> between in the allowed [low, high] range. In addition, there is a heuristic 
> that determines the positions at which the key has hex characters, and the 
> random key contains hex characters at those positions as well.
> Example output for the random read workload:
> Time: 120 sec, seek/sec: 8290, kv/sec: 30351, kv bytes/sec: 91868121, 
> blk/sec: 10147, unique keys: 232779

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5368) Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible in HBase installs

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5368:
-

Attachment: 5368.txt

Simple patch. Just moves/renames the class and adds safety checks.

> Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible 
> in HBase installs
> -
>
> Key: HBASE-5368
> URL: https://issues.apache.org/jira/browse/HBASE-5368
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 5368.txt
>
>
> Very simple change to make PrefixSplitKeyPolicy accessible in HBase installs 
> (user still needs to setup the table(s) accordingly).
> Right now it is in src/test/org.apache.hadoop.hbase.regionserver, I propose 
> moving it to src/org.apache.hadoop.hbase.regionserver (alongside 
> ConstantSizeRegionSplitPolicy), and maybe renaming it too.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4218) Data Block Encoding of KeyValues (aka delta encoding / prefix compression)

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205176#comment-13205176
 ] 

Hadoop QA commented on HBASE-4218:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514065/D1659.2.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 106 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/935//console

This message is automatically generated.

> Data Block Encoding of KeyValues  (aka delta encoding / prefix compression)
> ---
>
> Key: HBASE-4218
> URL: https://issues.apache.org/jira/browse/HBASE-4218
> Project: HBase
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 0.94.0
>Reporter: Jacek Migdal
>Assignee: Mikhail Bautin
>  Labels: compression
> Fix For: 0.94.0
>
> Attachments: 0001-Delta-encoding-fixed-encoded-scanners.patch, 
> 0001-Delta-encoding.patch, 4218-2012-01-14.txt, 4218-v16.txt, 4218.txt, 
> D1659.1.patch, D1659.2.patch, D447.1.patch, D447.10.patch, D447.11.patch, 
> D447.12.patch, D447.13.patch, D447.14.patch, D447.15.patch, D447.16.patch, 
> D447.17.patch, D447.18.patch, D447.19.patch, D447.2.patch, D447.20.patch, 
> D447.21.patch, D447.22.patch, D447.23.patch, D447.24.patch, D447.25.patch, 
> D447.26.patch, D447.3.patch, D447.4.patch, D447.5.patch, D447.6.patch, 
> D447.7.patch, D447.8.patch, D447.9.patch, 
> Data-block-encoding-2011-12-23.patch, 
> Delta-encoding-2012-01-17_11_09_09.patch, 
> Delta-encoding-2012-01-25_00_45_29.patch, 
> Delta-encoding-2012-01-25_16_32_14.patch, 
> Delta-encoding.patch-2011-12-22_11_52_07.patch, 
> Delta-encoding.patch-2012-01-05_15_16_43.patch, 
> Delta-encoding.patch-2012-01-05_16_31_44.patch, 
> Delta-encoding.patch-2012-01-05_16_31_44_copy.patch, 
> Delta-encoding.patch-2012-01-05_18_50_47.patch, 
> Delta-encoding.patch-2012-01-07_14_12_48.patch, 
> Delta-encoding.patch-2012-01-13_12_20_07.patch, 
> Delta_encoding_with_memstore_TS.patch, open-source.diff
>
>
> A compression for keys. Keys are sorted in HFile and they are usually very 
> similar. Because of that, it is possible to design better compression than 
> general purpose algorithms,
> It is an additional step designed to be used in memory. It aims to save 
> memory in cache as well as speeding seeks within HFileBlocks. It should 
> improve performance a lot, if key lengths are larger than value lengths. For 
> example, it makes a lot of sense to use it when value is a counter.
> Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) 
> shows that I could achieve decent level of compression:
>  key compression ratio: 92%
>  total compression ratio: 85%
>  LZO on the same data: 85%
>  LZO after delta encoding: 91%
> While having much better performance (20-80% faster decompression ratio than 
> LZO). Moreover, it should allow far more efficient seeking which should 
> improve performance a bit.
> It seems that a simple compression algorithms are good enough. Most of the 
> savings are due to prefix compression, int128 encoding, timestamp diffs and 
> bitfields to avoid duplication. That way, comparisons of compressed data can 
> be much faster than a byte comparator (thanks to prefix compression and 
> bitfields).
> In order to implement it in HBase two important changes in design will be 
> needed:
> -solidify interface to HFileBlock / HFileReader Scanner to provide seeking 
> and iterating; access to uncompressed buffer in HFileBlock will have bad 
> performance
> -extend comparators to support comparison assuming that N first bytes are 
> equal (or some fields are equal)
> Link to a discussion about something similar:
> http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windows&subj=Re+prefix+compression

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205171#comment-13205171
 ] 

Phabricator commented on HBASE-5199:


mbautin has commented on the revision "[jira][HBASE-5199] Delete out of TTL 
store files before compaction selection ".

  @Liyin: could you please post the rebased version of the patch so I can 
commit?

REVISION DETAIL
  https://reviews.facebook.net/D1311


> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205175#comment-13205175
 ] 

Phabricator commented on HBASE-5199:


mbautin has commented on the revision "[jira][HBASE-5199] Delete out of TTL 
store files before compaction selection ".

  @Liyin: could you please post the rebased version of the patch so I can 
commit?

REVISION DETAIL
  https://reviews.facebook.net/D1311


> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205174#comment-13205174
 ] 

Phabricator commented on HBASE-5199:


mbautin has commented on the revision "[jira][HBASE-5199] Delete out of TTL 
store files before compaction selection ".

  @Liyin: could you please post the rebased version of the patch so I can 
commit?

REVISION DETAIL
  https://reviews.facebook.net/D1311


> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205172#comment-13205172
 ] 

Phabricator commented on HBASE-5199:


mbautin has commented on the revision "[jira][HBASE-5199] Delete out of TTL 
store files before compaction selection ".

  @Liyin: could you please post the rebased version of the patch so I can 
commit?

REVISION DETAIL
  https://reviews.facebook.net/D1311


> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205173#comment-13205173
 ] 

Phabricator commented on HBASE-5199:


mbautin has commented on the revision "[jira][HBASE-5199] Delete out of TTL 
store files before compaction selection ".

  @Liyin: could you please post the rebased version of the patch so I can 
commit?

REVISION DETAIL
  https://reviews.facebook.net/D1311


> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4218) Data Block Encoding of KeyValues (aka delta encoding / prefix compression)

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-4218:
---

Attachment: D1659.2.patch

mbautin updated the revision "[jira] [HBASE-4218] [89-fb] Porting HFile data 
block encoding to 89-fb".
Reviewers: Kannan, Karthik, nspiegelberg, gqchen, JIRA

  Fixing DataBlockEncodingTool. Block-level compression parameter was not being 
handled correctly.

REVISION DETAIL
  https://reviews.facebook.net/D1659

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/KeyValue.java
  src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
  src/main/java/org/apache/hadoop/hbase/client/Result.java
  src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java
  
src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/CompressionState.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java
  
src/main/java/org/apache/hadoop/hbase/io/encoding/EncoderBufferTooSmallException.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheKey.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
  src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
  
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
  
src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java
  src/main/java/org/apache/hadoop/hbase/util/ByteBufferUtils.java
  src/main/java/org/apache/hadoop/hbase/util/CompoundBloomFilter.java
  src/main/ruby/hbase/admin.rb
  src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
  src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
  src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java
  src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
  src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
  src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
  src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java
  src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java
  src/test/java/org/apache/hadoop/hbase/io/encoding/RedundantKVGenerator.java
  
src/test/java/org/apache/hadoop/hbase/io/encoding/TestBufferedDataBlockEncoder.java
  src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java
  src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java
  src/test/java/org/apache/hadoop/hbase/io/encoding/TestEncodedSeekers.java
  
src/test/java/org/apache/hadoop/hbase/io/encoding/TestUpgradeFromHFileV1ToEncoding.java
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.j

[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205163#comment-13205163
 ] 

Hadoop QA commented on HBASE-5199:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514057/hbase-5199.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -136 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 156 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.replication.TestReplicationPeer
  org.apache.hadoop.hbase.io.hfile.TestHFileBlock
  org.apache.hadoop.hbase.TestDrainingServer
  org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/934//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/934//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/934//console

This message is automatically generated.

> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5199) Delete out of TTL store files before compaction selection

[Liyin Tang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liyin Tang updated HBASE-5199:
--

Attachment: hbase-5199.patch

The new patch is rebased on the latest trunk and all the unit tests are passed.

> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5199) Delete out of TTL store files before compaction selection

[Liyin Tang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liyin Tang updated HBASE-5199:
--

Attachment: (was: HBASE-5199.patch)

> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205139#comment-13205139
 ] 

stack commented on HBASE-5325:
--

Ok.  I can go along w/ cutting the scope of this issue back to its original 
intent (smile -- sorry for the hairbraining).

I was going to make a comment that we have the jmx 'model' not be a completely 
new one -- that it instead pick the existing model (though admittedly its a bit 
messy) -- but it looks like your hand has been forced some already.   I was 
going to suggest that we name the mbean for metrics MasterMetrics rather than 
MasterStatistics but I see this an existing MBean (Who named our mbean 'Master' 
and 'MasterStatistics' -- our history doesn't say... they don't seem like good 
names... why not org.apache.hbase prefix... etc.)

Why is HBaseMasterMXBean in hbase.metrics and not in hbase.master.metrics?  
Ditto HbaseRegionServerMXBean

You don't need these in your new files:

+ * Copyright 2012 The Apache Software Foundation

Why not name the bean for the master servername especially as there can be 
multiple masters running in a cluster -- getServerName.

Why would you have this info in regionserver jmx attributes:

+  public String getHBaseMaster();

This is pretty useless:

+  public long getStartCode();

Better return the regionserver ServerName.  Or name the mbean for the 
ServerName.

More to follow...



> Expose basic information about the master-status through jmx beans 
> ---
>
> Key: HBASE-5325
> URL: https://issues.apache.org/jira/browse/HBASE-5325
> Project: HBase
>  Issue Type: Improvement
>Reporter: Hitesh Shah
>Assignee: Hitesh Shah
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch
>
>
> Similar to the Namenode and Jobtracker, it would be good if the hbase master 
> could expose some information through mbeans.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5364) Fix source files missing licenses

[Elliott Clark (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205135#comment-13205135
 ] 

Elliott Clark commented on HBASE-5364:
--

Thanks

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch, hbase-5364-0.92.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205129#comment-13205129
 ] 

jirapos...@reviews.apache.org commented on HBASE-5358:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3811/#review4999
---



src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java


Ok.  Good.


- Michael


On 2012-02-09 22:18:10, enis wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3811/
bq.  ---
bq.  
bq.  (Updated 2012-02-09 22:18:10)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] 
where A extends Writable. This becomes an issue for example when adding a 
coprocessor method which takes A[] (see HBASE-5352).
bq.  
bq.  
bq.  This addresses bug HBASE-5358.
bq.  https://issues.apache.org/jira/browse/HBASE-5358
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
260f982 
bq.src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java 
78513ce 
bq.  
bq.  Diff: https://reviews.apache.org/r/3811/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  enis
bq.  
bq.



> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Affects Versions: 0.94.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5374) useTableNameGlobally is not initialized for ReaderV2

[Zhihong Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5374:
--

Affects Version/s: 0.94.0

> useTableNameGlobally is not initialized for ReaderV2
> 
>
> Key: HBASE-5374
> URL: https://issues.apache.org/jira/browse/HBASE-5374
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Zhihong Yu
>
> SchemaMetrics.useTableNameGlobally is a Boolean object that is not 
> initialized. It depends on public static method configureGlobally() to 
> initialize it based on the configuration file. But this is only done for 
> writer, not for reader. So when invoking hfile tool,
> {code}
> hbase/bin/hbase org.apache.hadoop.hbase.io.hfile.HFile -v -f YourFile
> {code}
> where HFileReaderV2 is invoked, it throws exception complaining the flag is 
> null. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5374) useTableNameGlobally is not initialized for ReaderV2

[Zhihong Yu (Created) (JIRA)]

useTableNameGlobally is not initialized for ReaderV2


 Key: HBASE-5374
 URL: https://issues.apache.org/jira/browse/HBASE-5374
 Project: HBase
  Issue Type: Bug
Reporter: Zhihong Yu


SchemaMetrics.useTableNameGlobally is a Boolean object that is not initialized. 
It depends on public static method configureGlobally() to initialize it based 
on the configuration file. But this is only done for writer, not for reader. So 
when invoking hfile tool,
{code}
hbase/bin/hbase org.apache.hadoop.hbase.io.hfile.HFile -v -f YourFile
{code}
where HFileReaderV2 is invoked, it throws exception complaining the flag is 
null. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5373) Table level lock to prevent the race of multiple table level operation

[Liyin Tang (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205120#comment-13205120
 ] 

Liyin Tang commented on HBASE-5373:
---

Cool! Sounds like what I try to do here. I will take a look over Accumulo.

> Table level lock to prevent the race of multiple table level operation
> --
>
> Key: HBASE-5373
> URL: https://issues.apache.org/jira/browse/HBASE-5373
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
>
> A table level lock can guarantee that only one table operation would happen 
> at one time for each table. The master should require and release these table 
> locks correctly during the failover time. One proposal is to keep track of 
> the lock and its corresponding operation in the zookeeper. If there is a 
> master failover, the secondary should have a way to check whether these 
> operations are succeeded nor not before releasing the lock.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5373) Table level lock to prevent the race of multiple table level operation

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205108#comment-13205108
 ] 

Todd Lipcon commented on HBASE-5373:


Over in Accumulo land they have a neat framework called "FATE" (FAult Tolerant 
Execution or something?) Basically, they put up a znode in ZK for any 
master-coordinated action, and it acts as an "intent log" of the idempotent 
steps. If there's a failover, the new master will finish off any 
already-running FATE transaction.

> Table level lock to prevent the race of multiple table level operation
> --
>
> Key: HBASE-5373
> URL: https://issues.apache.org/jira/browse/HBASE-5373
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
>
> A table level lock can guarantee that only one table operation would happen 
> at one time for each table. The master should require and release these table 
> locks correctly during the failover time. One proposal is to keep track of 
> the lock and its corresponding operation in the zookeeper. If there is a 
> master failover, the secondary should have a way to check whether these 
> operations are succeeded nor not before releasing the lock.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205093#comment-13205093
 ] 

Hadoop QA commented on HBASE-5363:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514046/hbase-5363.2.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 javadoc.  The javadoc tool appears to have generated -136 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 156 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.io.hfile.TestHFileBlock

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/933//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/933//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/933//console

This message is automatically generated.

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363-0.90.patch, hbase-5363.2.patch, 
> hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5364) Fix source files missing licenses

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205089#comment-13205089
 ] 

Jonathan Hsieh commented on HBASE-5364:
---

Found an example of debian control files elsewhere where commented license 
included.  Will include.

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch, hbase-5364-0.92.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205080#comment-13205080
 ] 

Jonathan Hsieh commented on HBASE-5327:
---

My comment was really just asking for more info in the first version's IAE 
string. 


> Print a message when an invalid hbase.rootdir is passed
> ---
>
> Key: HBASE-5327
> URL: https://issues.apache.org/jira/browse/HBASE-5327
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: Jean-Daniel Cryans
>Assignee: Jimmy Xiang
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: hbase-5327.txt, hbase-5327_v2.txt
>
>
> As seen on the mailing list: 
> http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
> If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a 
> path to .oldlogs:
> {noformat}
> 2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster: 
> Unhandled exception. Starting shutdown.
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
> at org.apache.hadoop.fs.Path.initialize(Path.java:148)
> at org.apache.hadoop.fs.Path.(Path.java:71)
> at org.apache.hadoop.fs.Path.(Path.java:50)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:112)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> hdfs://sv4r11s38:9100.oldlogs
> at java.net.URI.checkPath(URI.java:1787)
> at java.net.URI.(URI.java:735)
> at org.apache.hadoop.fs.Path.initialize(Path.java:145)
> ... 6 more
> {noformat}
> It could also crash anywhere else, this just happens to be the first place we 
> use hbase.rootdir. We need to verify that it's an actual folder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5364) Fix source files missing licenses

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5364:
--

Attachment: hbase-5364-0.92.patch

Patch for 0.92 license violation.

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch, hbase-5364-0.92.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5373) Table level lock to prevent the race of multiple table level operation

[Liyin Tang (Created) (JIRA)]

Table level lock to prevent the race of multiple table level operation
--

 Key: HBASE-5373
 URL: https://issues.apache.org/jira/browse/HBASE-5373
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang
Assignee: Liyin Tang


A table level lock can guarantee that only one table operation would happen at 
one time for each table. The master should require and release these table 
locks correctly during the failover time. One proposal is to keep track of the 
lock and its corresponding operation in the zookeeper. If there is a master 
failover, the secondary should have a way to check whether these operations are 
succeeded nor not before releasing the lock.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5364) Fix source files missing licenses

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205073#comment-13205073
 ] 

Jonathan Hsieh commented on HBASE-5364:
---

Not clear to me if src/packages/deb/conf-pseudo.control/control needs to have 
license or not (and don't know how to test).  I may exclude that one the trunk 
version.

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5364) Fix source files missing licenses

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205072#comment-13205072
 ] 

Jonathan Hsieh commented on HBASE-5364:
---

Not clear to me if src/packages/deb/conf-pseudo.control/control needs to have 
license or not (and don't know how to test).  I may exclude that one the trunk 
version.

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5364) Fix source files missing licenses

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205063#comment-13205063
 ] 

Jonathan Hsieh commented on HBASE-5364:
---

I'll just take care of it before I commit.

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed

[Jimmy Xiang (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205064#comment-13205064
 ] 

Jimmy Xiang commented on HBASE-5327:


I prefer the first version actually.  If the root dir is invalid, HDFS will 
throw an IAE. That's how we know a path an invalid HDFS path.

> Print a message when an invalid hbase.rootdir is passed
> ---
>
> Key: HBASE-5327
> URL: https://issues.apache.org/jira/browse/HBASE-5327
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: Jean-Daniel Cryans
>Assignee: Jimmy Xiang
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: hbase-5327.txt, hbase-5327_v2.txt
>
>
> As seen on the mailing list: 
> http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
> If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a 
> path to .oldlogs:
> {noformat}
> 2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster: 
> Unhandled exception. Starting shutdown.
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
> at org.apache.hadoop.fs.Path.initialize(Path.java:148)
> at org.apache.hadoop.fs.Path.(Path.java:71)
> at org.apache.hadoop.fs.Path.(Path.java:50)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:112)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> hdfs://sv4r11s38:9100.oldlogs
> at java.net.URI.checkPath(URI.java:1787)
> at java.net.URI.(URI.java:735)
> at org.apache.hadoop.fs.Path.initialize(Path.java:145)
> ... 6 more
> {noformat}
> It could also crash anywhere else, this just happens to be the first place we 
> use hbase.rootdir. We need to verify that it's an actual folder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5364) Fix source files missing licenses

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205059#comment-13205059
 ] 

Jonathan Hsieh commented on HBASE-5364:
---

Elliot, After your patch and HBASE-5363, there is one other violation on trunk: 
 bin/hbase-jruby 

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205056#comment-13205056
 ] 

Zhihong Yu commented on HBASE-5358:
---

Test result looks good.

Plan to integrate the patch tomorrow.

> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Affects Versions: 0.94.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205053#comment-13205053
 ] 

Hadoop QA commented on HBASE-5358:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514031/HBASE-5358_v3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 11 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -136 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 156 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/932//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/932//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/932//console

This message is automatically generated.

> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Affects Versions: 0.94.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205049#comment-13205049
 ] 

Jonathan Hsieh commented on HBASE-5327:
---

I'm not a fan of using the IAE for control flow, but the lgtm.

> Print a message when an invalid hbase.rootdir is passed
> ---
>
> Key: HBASE-5327
> URL: https://issues.apache.org/jira/browse/HBASE-5327
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: Jean-Daniel Cryans
>Assignee: Jimmy Xiang
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: hbase-5327.txt, hbase-5327_v2.txt
>
>
> As seen on the mailing list: 
> http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
> If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a 
> path to .oldlogs:
> {noformat}
> 2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster: 
> Unhandled exception. Starting shutdown.
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
> at org.apache.hadoop.fs.Path.initialize(Path.java:148)
> at org.apache.hadoop.fs.Path.(Path.java:71)
> at org.apache.hadoop.fs.Path.(Path.java:50)
> at 
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:112)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> hdfs://sv4r11s38:9100.oldlogs
> at java.net.URI.checkPath(URI.java:1787)
> at java.net.URI.(URI.java:735)
> at org.apache.hadoop.fs.Path.initialize(Path.java:145)
> ... 6 more
> {noformat}
> It could also crash anywhere else, this just happens to be the first place we 
> use hbase.rootdir. We need to verify that it's an actual folder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5363:
--

Status: Patch Available  (was: Open)

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.92.0, 0.90.5
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363-0.90.patch, hbase-5363.2.patch, 
> hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5363:
--

Attachment: hbase-5363.2.patch

v2 of the trunk/0.92 patch. v1 likely breaks website report generation.

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363-0.90.patch, hbase-5363.2.patch, 
> hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5363:
--

Attachment: hbase-5363-0.90.patch

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363-0.90.patch, hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5372) Table mutation operations should check table level rights, not global rights

[Enis Soztutar (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205035#comment-13205035
 ] 

Enis Soztutar commented on HBASE-5372:
--

Replicating Andrew's comments on the parent issue:
bq. This certainly makes sense for grant/revoke.
bq. However, creating, dropping, or modifying a table has global ramifications 
on the cluster. When making changes here it should remain possible to require 
global CREATE/ADMIN rights for those actions by policy. This way a site admin 
can prevent users from taking actions that perturb region assignment if desired.

Agreed on the possible affects on the global state of the cluster for some 
table level operations. If we do change the enforcement to the table level 
though, cluster admins can still opt to not give admin rights on the table 
level, thus they can prevent possibly disruptive operations. So this change 
will only give more flexibility, but will also allow the current setup, where 
ADMIN permissions are set only globally. 

However, for this maybe we have to revisit table ownership rights. Currently, 
the table owner has every right on the table, and this is not managed through 
the normal grant/revoke operations, but on the table metadata. We may want to 
remove table ownership, but introduce default table creation rights, which 
means that when a user creates a table, she automatically get those rights 
allocated. But another user can grant extra rights, or revoke them. 

> Table mutation operations should check table level rights, not global rights 
> -
>
> Key: HBASE-5372
> URL: https://issues.apache.org/jira/browse/HBASE-5372
> Project: HBase
>  Issue Type: Sub-task
>  Components: security
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> getUserPermissions(tableName)/grant/revoke and drop/modify table operations 
> should not check for global CREATE/ADMIN rights, but table CREATE/ADMIN 
> rights. The reasoning is that if a user is able to admin or read from a 
> table, she should be able to read the table's permissions. We can choose 
> whether we want only READ or ADMIN permissions for getUserPermission(). Since 
> we check for global permissions first for table permissions, configuring 
> table access using global permissions will continue to work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5342) Grant/Revoke global permissions

[Enis Soztutar (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-5342:
-

Issue Type: Sub-task  (was: Improvement)
Parent: HBASE-5352

> Grant/Revoke global permissions
> ---
>
> Key: HBASE-5342
> URL: https://issues.apache.org/jira/browse/HBASE-5342
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> HBASE-3025 introduced simple ACLs based on coprocessors. It defines 
> global/table/cf/cq level permissions. However, there is no way to 
> grant/revoke global level permissions, other than the hbase.superuser conf 
> setting. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205033#comment-13205033
 ] 

Jonathan Hsieh commented on HBASE-5363:
---

made a 0.90 version, reports about 40 errors (probably similar to the 0.92 
branch errors)

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205023#comment-13205023
 ] 

Jonathan Hsieh commented on HBASE-5363:
---

patch applies on 0.92, reports 1 error (src/docbkx/build.xml)


> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5372) Table mutation operations should check table level rights, not global rights

[Enis Soztutar (Created) (JIRA)]

Table mutation operations should check table level rights, not global rights 
-

 Key: HBASE-5372
 URL: https://issues.apache.org/jira/browse/HBASE-5372
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar


getUserPermissions(tableName)/grant/revoke and drop/modify table operations 
should not check for global CREATE/ADMIN rights, but table CREATE/ADMIN rights. 
The reasoning is that if a user is able to admin or read from a table, she 
should be able to read the table's permissions. We can choose whether we want 
only READ or ADMIN permissions for getUserPermission(). Since we check for 
global permissions first for table permissions, configuring table access using 
global permissions will continue to work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205021#comment-13205021
 ] 

jirapos...@reviews.apache.org commented on HBASE-5371:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3829/
---

Review request for hbase.


Summary
---

We need to introduce something like 
AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
that clients can check access rights before carrying out the operations. We 
need this kind of operation for HCATALOG-245, which introduces authorization 
providers for hbase over hcat. We cannot use getUserPermissions() since it 
requires ADMIN permissions on the global/table level.


This addresses bug HBASE-5371.
https://issues.apache.org/jira/browse/HBASE-5371


Diffs
-

  
security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
 5091b7d 
  
security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
 5fa2edb 
  
security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 f864373 

Diff: https://reviews.apache.org/r/3829/diff


Testing
---


Thanks,

enis



> Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) 
> API
> 
>
> Key: HBASE-5371
> URL: https://issues.apache.org/jira/browse/HBASE-5371
> Project: HBase
>  Issue Type: Sub-task
>  Components: security
>Affects Versions: 0.94.0, 0.92.1
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> We need to introduce something like 
> AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
> that clients can check access rights before carrying out the operations. We 
> need this kind of operation for HCATALOG-245, which introduces authorization 
> providers for hbase over hcat. We cannot use getUserPermissions() since it 
> requires ADMIN permissions on the global/table level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API

[Enis Soztutar (Created) (JIRA)]

Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API


 Key: HBASE-5371
 URL: https://issues.apache.org/jira/browse/HBASE-5371
 Project: HBase
  Issue Type: Sub-task
  Components: security
Affects Versions: 0.94.0, 0.92.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar


We need to introduce something like 
AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
that clients can check access rights before carrying out the operations. We 
need this kind of operation for HCATALOG-245, which introduces authorization 
providers for hbase over hcat. We cannot use getUserPermissions() since it 
requires ADMIN permissions on the global/table level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205011#comment-13205011
 ] 

Jonathan Hsieh commented on HBASE-5363:
---

@Elliott, since you posted on the other jira, I deleted the attachment that you 
placed here.

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5363:
--

Attachment: (was: HBASE-5363-1.patch)

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5363:
--

Attachment: hbase-5363.patch

This patch adds rat checks to the package phase of the mvn build when the 
release profile is activated (-Prelease).  

It cleans up some of the rat excludes (ignore auto gen stuff), and at least on 
trunk id's 5 legit license violations.

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: HBASE-5363-1.patch, hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205004#comment-13205004
 ] 

Jonathan Hsieh commented on HBASE-5363:
---

To run, us 'mvn package -Prelease -DskipTests'.  Note that the excludes are not 
used if you use 'mvn rat:check'.   I need help from a mavenista to figure out 
how to make that work without copy paste.

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: HBASE-5363-1.patch, hbase-5363.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5364) Fix source files missing licenses

[Elliott Clark (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205002#comment-13205002
 ] 

Elliott Clark commented on HBASE-5364:
--

Attach patch to fix files on master.

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server

[Mubarak Seyed (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204998#comment-13204998
 ] 

Mubarak Seyed commented on HBASE-4720:
--

The following scenarios are tested:

1. 5 nodes dev cluster, running trunk
   * Generated load with curl for checkAndPut/checkAndDelete, tested for 
functional verification
   * Developed a test client, which uses HBaseAdmin APIs to test the atomic 
operations functionality (for   positive/negative testcases)

2. 7 nodes perf cluster, running trunk
   * Developed a test client, which uses HBaseAdmin APIs to test the atomic 
operations functionality (for positive/negative testcases).

> Implement atomic update operations (checkAndPut, checkAndDelete) for REST 
> client/server 
> 
>
> Key: HBASE-4720
> URL: https://issues.apache.org/jira/browse/HBASE-4720
> Project: HBase
>  Issue Type: Improvement
>Reporter: Daniel Lord
>Assignee: Mubarak Seyed
> Fix For: 0.94.0
>
> Attachments: HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, 
> HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, 
> HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, 
> HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch
>
>
> I have several large application/HBase clusters where an application node 
> will occasionally need to talk to HBase from a different cluster.  In order 
> to help ensure some of my consistency guarantees I have a sentinel table that 
> is updated atomically as users interact with the system.  This works quite 
> well for the "regular" hbase client but the REST client does not implement 
> the checkAndPut and checkAndDelete operations.  This exposes the application 
> to some race conditions that have to be worked around.  It would be ideal if 
> the same checkAndPut/checkAndDelete operations could be supported by the REST 
> client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5363) Add rat check run automatically on mvn release build.

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5363:
--

Summary: Add rat check run automatically on mvn release build.  (was: Add 
rat check to run automatically on mvn build.)

> Add rat check run automatically on mvn release build.
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: HBASE-5363-1.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5370) Allow HBase shell to set HTableDescriptor values

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5370:
-

Summary: Allow HBase shell to set HTableDescriptor values  (was: Allow 
HBase shell set HTableDescriptor values)

> Allow HBase shell to set HTableDescriptor values
> 
>
> Key: HBASE-5370
> URL: https://issues.apache.org/jira/browse/HBASE-5370
> Project: HBase
>  Issue Type: Improvement
>Reporter: Lars Hofhansl
>Priority: Minor
>
> Currently it does not seem to be possible to set value on a table's 
> HTableDescriptor (either on creation or afterwards).
> The syntax I have in mind is something like:
> create {NAME=>'table', 'somekey'=>'somevalue'}, 'column'
> In analogy to how we allow a column to either a string ('column') or an 
> association {NAME=>'column', ...}
> alter would be changed to allow setting arbitrary values.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5363:
--

Summary: Automatically run rat check on mvn release builds  (was: Add rat 
check run automatically on mvn release build.)

> Automatically run rat check on mvn release builds
> -
>
> Key: HBASE-5363
> URL: https://issues.apache.org/jira/browse/HBASE-5363
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: HBASE-5363-1.patch
>
>
> Some of the recent hbase release failed rat checks (mvn rat:check).  We 
> should add checks likely in the mvn package phase so that this becomes a 
> non-issue in the future.
> Here's an example from Whirr:
> https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5370) Allow HBase shell set HTableDescriptor values

[Lars Hofhansl (Created) (JIRA)]

Allow HBase shell set HTableDescriptor values
-

 Key: HBASE-5370
 URL: https://issues.apache.org/jira/browse/HBASE-5370
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Priority: Minor


Currently it does not seem to be possible to set value on a table's 
HTableDescriptor (either on creation or afterwards).

The syntax I have in mind is something like:
create {NAME=>'table', 'somekey'=>'somevalue'}, 'column'

In analogy to how we allow a column to either a string ('column') or an 
association {NAME=>'column', ...}

alter would be changed to allow setting arbitrary values.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[Enis Soztutar (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-5358:
-

Affects Version/s: 0.94.0
   Status: Patch Available  (was: Open)

> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Affects Versions: 0.94.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[Enis Soztutar (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-5358:
-

Attachment: HBASE-5358_v3.patch

Attaching the latest version of the patch from review. Incorporates a trivial 
javadoc fix as suggested. 

> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: HBASE-5358_v3.patch
>
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5364) Fix source files missing licenses

[Elliott Clark (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-5364:
-

Attachment: HBASE-5364-1.patch

> Fix source files missing licenses
> -
>
> Key: HBASE-5364
> URL: https://issues.apache.org/jira/browse/HBASE-5364
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.0, 0.90.5, 0.92.0
>Reporter: Jonathan Hsieh
>Priority: Blocker
> Attachments: HBASE-5364-1.patch
>
>
> running 'mvn rat:check' shows that a few files have snuck in that do not have 
> proper apache licenses.  Ideally we should fix these before we cut another 
> release/release candidate.
> This is a blocker for 0.94, and probably should be for the other branches as 
> well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5367) [book] small formatting changes to compaction description in Arch/Regions/Store

[Doug Meil (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204979#comment-13204979
 ] 

Doug Meil commented on HBASE-5367:
--

Should I change this back to 64M?


> [book] small formatting changes to compaction description in 
> Arch/Regions/Store
> ---
>
> Key: HBASE-5367
> URL: https://issues.apache.org/jira/browse/HBASE-5367
> Project: HBase
>  Issue Type: Improvement
>Reporter: Doug Meil
>Assignee: Doug Meil
>Priority: Minor
> Attachments: book_hbase_5367.xml.patch, book_hbase_5367_2.xml.patch
>
>
> Fixing a few small-but-important things that came out of a post-commit 
> comment in HBASE-5365
> book.xml
> * corrected default region flush size (it's actually 64mb)
> * removed trailing 'F' in a ratio discussion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5369) Compaction selection based on the hotness of the HFile's block in the block cache

[Liyin Tang (Created) (JIRA)]

Compaction selection based on the hotness of the HFile's block in the block 
cache
-

 Key: HBASE-5369
 URL: https://issues.apache.org/jira/browse/HBASE-5369
 Project: HBase
  Issue Type: Improvement
Reporter: Liyin Tang
Assignee: Liyin Tang


HBase reserves a large set memory for the block cache and the cached blocks 
will be age out in a LRU fashion. Obviously, we don't want to age out the 
blocks which are still hot. However, when the compactions are starting, these 
hot blocks may naturally be invalid. Considering that the block cache has 
already known which HFiles these hot blocks come from, the compaction selection 
algorithm could just simply skip compact these HFiles until these block cache 
become cold.
Furthermore, the HBase could compact multiple HFiles into two HFiles. One of 
them only contains hot blocks which are supposed be cached directly.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204969#comment-13204969
 ] 

jirapos...@reviews.apache.org commented on HBASE-5358:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3811/#review4995
---


Looks good.
Please attach patch to JIRA so that Hadoop QA can run test suite.


src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java


code is no longer a byte.
Please adjust javadoc accordingly.


- Ted


On 2012-02-09 22:18:10, enis wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3811/
bq.  ---
bq.  
bq.  (Updated 2012-02-09 22:18:10)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] 
where A extends Writable. This becomes an issue for example when adding a 
coprocessor method which takes A[] (see HBASE-5352).
bq.  
bq.  
bq.  This addresses bug HBASE-5358.
bq.  https://issues.apache.org/jira/browse/HBASE-5358
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
260f982 
bq.src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java 
78513ce 
bq.  
bq.  Diff: https://reviews.apache.org/r/3811/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  enis
bq.  
bq.



> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204956#comment-13204956
 ] 

jirapos...@reviews.apache.org commented on HBASE-5358:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3811/
---

(Updated 2012-02-09 22:18:10.009110)


Review request for hbase.


Changes
---

Incoprorated review comments, optimized the serialization of Arrays of know 
types, and added a few more test cases


Summary
---

HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A 
extends Writable. This becomes an issue for example when adding a coprocessor 
method which takes A[] (see HBASE-5352).


This addresses bug HBASE-5358.
https://issues.apache.org/jira/browse/HBASE-5358


Diffs (updated)
-

  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 260f982 
  src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java 78513ce 

Diff: https://reviews.apache.org/r/3811/diff


Testing
---


Thanks,

enis



> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3134) [replication] Add the ability to enable/disable streams

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204948#comment-13204948
 ] 

Hadoop QA commented on HBASE-3134:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514010/3134-v2.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 6 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated -136 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 156 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.io.hfile.TestHFileBlock
  org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/931//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/931//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/931//console

This message is automatically generated.

> [replication] Add the ability to enable/disable streams
> ---
>
> Key: HBASE-3134
> URL: https://issues.apache.org/jira/browse/HBASE-3134
> Project: HBase
>  Issue Type: New Feature
>  Components: replication
>Reporter: Jean-Daniel Cryans
>Assignee: Teruyoshi Zenmyo
>Priority: Minor
>  Labels: replication
> Fix For: 0.94.0
>
> Attachments: 3134-v2.txt, 3134.txt, HBASE-3134.patch, 
> HBASE-3134.patch, HBASE-3134.patch, HBASE-3134.patch
>
>
> This jira was initially in the scope of HBASE-2201, but was pushed out since 
> it has low value compared to the required effort (and when want to ship 
> 0.90.0 rather soonish).
> We need to design a way to enable/disable replication streams in a 
> determinate fashion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204943#comment-13204943
 ] 

jirapos...@reviews.apache.org commented on HBASE-5358:
--



bq.  On 2012-02-09 02:16:29, Ted Yu wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 
606
bq.  > 
bq.  >
bq.  > Should we validate length ?

It is not validated in other array/list read operations. The allocation will 
fail anyway if the length is off, no? 


bq.  On 2012-02-09 02:16:29, Ted Yu wrote:
bq.  > src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java, 
line 246
bq.  > 
bq.  >
bq.  > What does InSeq mean ?

Oh, it comes from Seg from Scala, where Array and List extends. I'll 
change the method name to testPolymorphismInSequences(), and add some comments, 
if you don't have any suggestion. 


bq.  On 2012-02-09 02:16:29, Ted Yu wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 
445
bq.  > 
bq.  >
bq.  > What if componentType is another Array ?

The test case for A[][] already covers for this i believe. 


- enis


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3811/#review4952
---


On 2012-02-09 01:38:18, enis wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3811/
bq.  ---
bq.  
bq.  (Updated 2012-02-09 01:38:18)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] 
where A extends Writable. This becomes an issue for example when adding a 
coprocessor method which takes A[] (see HBASE-5352).
bq.  
bq.  
bq.  This addresses bug HBASE-5358.
bq.  https://issues.apache.org/jira/browse/HBASE-5358
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java 
78513ce 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
260f982 
bq.  
bq.  Diff: https://reviews.apache.org/r/3811/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  enis
bq.  
bq.



> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204942#comment-13204942
 ] 

jirapos...@reviews.apache.org commented on HBASE-5358:
--



bq.  On 2012-02-09 07:02:48, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 
446
bq.  > 
bq.  >
bq.  > We are changing what is on the line now when we serialize?  All 
arrays now serialize their type ahead of array length?  In our public API what 
arrays will this effect?  Do you know?  Does it require our upping the version 
on the Interface that uses HBaseObjectWritable?

We are not changing the serialized bytes for the arrays that are already in 
CLASS_TO_CODE. This only affects the arrays that are not defined there. For 
them, when you serialize, the deserializer throws an erroneous 
ClassNotFoundException. So, I don't think wire compatibility will be affected. 


- enis


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3811/#review4960
---


On 2012-02-09 01:38:18, enis wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3811/
bq.  ---
bq.  
bq.  (Updated 2012-02-09 01:38:18)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] 
where A extends Writable. This becomes an issue for example when adding a 
coprocessor method which takes A[] (see HBASE-5352).
bq.  
bq.  
bq.  This addresses bug HBASE-5358.
bq.  https://issues.apache.org/jira/browse/HBASE-5358
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java 
78513ce 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
260f982 
bq.  
bq.  Diff: https://reviews.apache.org/r/3811/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  enis
bq.  
bq.



> HBaseObjectWritable should be able to serialize generic arrays not defined 
> previously
> -
>
> Key: HBASE-5358
> URL: https://issues.apache.org/jira/browse/HBASE-5358
> Project: HBase
>  Issue Type: Improvement
>  Components: coprocessors, io
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
> A extends Writable. This becomes an issue for example when adding a 
> coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204934#comment-13204934
 ] 

Zhihong Yu commented on HBASE-5199:
---

HBASE-5199.patch doesn't apply cleanly:
{code}
3 out of 6 hunks FAILED -- saving rejects to file 
src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java.rej
patching file 
src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file 
src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java.rej
{code}
Can you rebase patch ?

> Delete out of TTL store files before compaction selection
> -
>
> Key: HBASE-5199
> URL: https://issues.apache.org/jira/browse/HBASE-5199
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: Liyin Tang
> Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, 
> D1311.4.patch, D1311.5.patch, D1311.5.patch, HBASE-5199.patch
>
>
> Currently, HBase deletes the out of TTL store files after compaction. We can 
> change the sequence to delete the out of TTL store files before selecting 
> store files for compactions. 
> In this way, HBase can keep deleting the old invalid store files without 
> compaction, and also prevent from unnecessary compactions since the out of 
> TTL store files will be deleted before the compaction selection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira