[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5363: -- Attachment: (was: HBASE-5363-1.patch) Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205011#comment-13205011 ] Jonathan Hsieh commented on HBASE-5363: --- @Elliott, since you posted on the other jira, I deleted the attachment that you placed here. Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API
Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API Key: HBASE-5371 URL: https://issues.apache.org/jira/browse/HBASE-5371 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0, 0.92.1 Reporter: Enis Soztutar Assignee: Enis Soztutar We need to introduce something like AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so that clients can check access rights before carrying out the operations. We need this kind of operation for HCATALOG-245, which introduces authorization providers for hbase over hcat. We cannot use getUserPermissions() since it requires ADMIN permissions on the global/table level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API
[ https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205021#comment-13205021 ] jirapos...@reviews.apache.org commented on HBASE-5371: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3829/ --- Review request for hbase. Summary --- We need to introduce something like AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so that clients can check access rights before carrying out the operations. We need this kind of operation for HCATALOG-245, which introduces authorization providers for hbase over hcat. We cannot use getUserPermissions() since it requires ADMIN permissions on the global/table level. This addresses bug HBASE-5371. https://issues.apache.org/jira/browse/HBASE-5371 Diffs - security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java 5091b7d security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java 5fa2edb security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java f864373 Diff: https://reviews.apache.org/r/3829/diff Testing --- Thanks, enis Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API Key: HBASE-5371 URL: https://issues.apache.org/jira/browse/HBASE-5371 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0, 0.92.1 Reporter: Enis Soztutar Assignee: Enis Soztutar We need to introduce something like AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so that clients can check access rights before carrying out the operations. We need this kind of operation for HCATALOG-245, which introduces authorization providers for hbase over hcat. We cannot use getUserPermissions() since it requires ADMIN permissions on the global/table level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5372) Table mutation operations should check table level rights, not global rights
Table mutation operations should check table level rights, not global rights - Key: HBASE-5372 URL: https://issues.apache.org/jira/browse/HBASE-5372 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar getUserPermissions(tableName)/grant/revoke and drop/modify table operations should not check for global CREATE/ADMIN rights, but table CREATE/ADMIN rights. The reasoning is that if a user is able to admin or read from a table, she should be able to read the table's permissions. We can choose whether we want only READ or ADMIN permissions for getUserPermission(). Since we check for global permissions first for table permissions, configuring table access using global permissions will continue to work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205023#comment-13205023 ] Jonathan Hsieh commented on HBASE-5363: --- patch applies on 0.92, reports 1 error (src/docbkx/build.xml) Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205033#comment-13205033 ] Jonathan Hsieh commented on HBASE-5363: --- made a 0.90 version, reports about 40 errors (probably similar to the 0.92 branch errors) Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5372) Table mutation operations should check table level rights, not global rights
[ https://issues.apache.org/jira/browse/HBASE-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205035#comment-13205035 ] Enis Soztutar commented on HBASE-5372: -- Replicating Andrew's comments on the parent issue: bq. This certainly makes sense for grant/revoke. bq. However, creating, dropping, or modifying a table has global ramifications on the cluster. When making changes here it should remain possible to require global CREATE/ADMIN rights for those actions by policy. This way a site admin can prevent users from taking actions that perturb region assignment if desired. Agreed on the possible affects on the global state of the cluster for some table level operations. If we do change the enforcement to the table level though, cluster admins can still opt to not give admin rights on the table level, thus they can prevent possibly disruptive operations. So this change will only give more flexibility, but will also allow the current setup, where ADMIN permissions are set only globally. However, for this maybe we have to revisit table ownership rights. Currently, the table owner has every right on the table, and this is not managed through the normal grant/revoke operations, but on the table metadata. We may want to remove table ownership, but introduce default table creation rights, which means that when a user creates a table, she automatically get those rights allocated. But another user can grant extra rights, or revoke them. Table mutation operations should check table level rights, not global rights - Key: HBASE-5372 URL: https://issues.apache.org/jira/browse/HBASE-5372 Project: HBase Issue Type: Sub-task Components: security Reporter: Enis Soztutar Assignee: Enis Soztutar getUserPermissions(tableName)/grant/revoke and drop/modify table operations should not check for global CREATE/ADMIN rights, but table CREATE/ADMIN rights. The reasoning is that if a user is able to admin or read from a table, she should be able to read the table's permissions. We can choose whether we want only READ or ADMIN permissions for getUserPermission(). Since we check for global permissions first for table permissions, configuring table access using global permissions will continue to work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5363: -- Attachment: hbase-5363-0.90.patch Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363-0.90.patch, hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5363: -- Attachment: hbase-5363.2.patch v2 of the trunk/0.92 patch. v1 likely breaks website report generation. Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363-0.90.patch, hbase-5363.2.patch, hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5363: -- Status: Patch Available (was: Open) Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.92.0, 0.90.5 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363-0.90.patch, hbase-5363.2.patch, hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed
[
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205049#comment-13205049
]
Jonathan Hsieh commented on HBASE-5327:
---
I'm not a fan of using the IAE for control flow, but the lgtm.
Print a message when an invalid hbase.rootdir is passed
---
Key: HBASE-5327
URL: https://issues.apache.org/jira/browse/HBASE-5327
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jimmy Xiang
Fix For: 0.94.0, 0.90.7, 0.92.1
Attachments: hbase-5327.txt, hbase-5327_v2.txt
As seen on the mailing list:
http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a
path to .oldlogs:
{noformat}
2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
at org.apache.hadoop.fs.Path.initialize(Path.java:148)
at org.apache.hadoop.fs.Path.init(Path.java:71)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
hdfs://sv4r11s38:9100.oldlogs
at java.net.URI.checkPath(URI.java:1787)
at java.net.URI.init(URI.java:735)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
... 6 more
{noformat}
It could also crash anywhere else, this just happens to be the first place we
use hbase.rootdir. We need to verify that it's an actual folder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously
[ https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205053#comment-13205053 ] Hadoop QA commented on HBASE-5358: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514031/HBASE-5358_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 156 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/932//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/932//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/932//console This message is automatically generated. HBaseObjectWritable should be able to serialize generic arrays not defined previously - Key: HBASE-5358 URL: https://issues.apache.org/jira/browse/HBASE-5358 Project: HBase Issue Type: Improvement Components: coprocessors, io Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HBASE-5358_v3.patch HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously
[ https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205056#comment-13205056 ] Zhihong Yu commented on HBASE-5358: --- Test result looks good. Plan to integrate the patch tomorrow. HBaseObjectWritable should be able to serialize generic arrays not defined previously - Key: HBASE-5358 URL: https://issues.apache.org/jira/browse/HBASE-5358 Project: HBase Issue Type: Improvement Components: coprocessors, io Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HBASE-5358_v3.patch HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205059#comment-13205059 ] Jonathan Hsieh commented on HBASE-5364: --- Elliot, After your patch and HBASE-5363, there is one other violation on trunk: bin/hbase-jruby Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205063#comment-13205063 ] Jonathan Hsieh commented on HBASE-5364: --- I'll just take care of it before I commit. Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed
[
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205064#comment-13205064
]
Jimmy Xiang commented on HBASE-5327:
I prefer the first version actually. If the root dir is invalid, HDFS will
throw an IAE. That's how we know a path an invalid HDFS path.
Print a message when an invalid hbase.rootdir is passed
---
Key: HBASE-5327
URL: https://issues.apache.org/jira/browse/HBASE-5327
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jimmy Xiang
Fix For: 0.94.0, 0.90.7, 0.92.1
Attachments: hbase-5327.txt, hbase-5327_v2.txt
As seen on the mailing list:
http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a
path to .oldlogs:
{noformat}
2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
at org.apache.hadoop.fs.Path.initialize(Path.java:148)
at org.apache.hadoop.fs.Path.init(Path.java:71)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
hdfs://sv4r11s38:9100.oldlogs
at java.net.URI.checkPath(URI.java:1787)
at java.net.URI.init(URI.java:735)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
... 6 more
{noformat}
It could also crash anywhere else, this just happens to be the first place we
use hbase.rootdir. We need to verify that it's an actual folder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205073#comment-13205073 ] Jonathan Hsieh commented on HBASE-5364: --- Not clear to me if src/packages/deb/conf-pseudo.control/control needs to have license or not (and don't know how to test). I may exclude that one the trunk version. Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205072#comment-13205072 ] Jonathan Hsieh commented on HBASE-5364: --- Not clear to me if src/packages/deb/conf-pseudo.control/control needs to have license or not (and don't know how to test). I may exclude that one the trunk version. Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5364: -- Attachment: hbase-5364-0.92.patch Patch for 0.92 license violation. Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch, hbase-5364-0.92.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5373) Table level lock to prevent the race of multiple table level operation
Table level lock to prevent the race of multiple table level operation -- Key: HBASE-5373 URL: https://issues.apache.org/jira/browse/HBASE-5373 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang A table level lock can guarantee that only one table operation would happen at one time for each table. The master should require and release these table locks correctly during the failover time. One proposal is to keep track of the lock and its corresponding operation in the zookeeper. If there is a master failover, the secondary should have a way to check whether these operations are succeeded nor not before releasing the lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed
[
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205080#comment-13205080
]
Jonathan Hsieh commented on HBASE-5327:
---
My comment was really just asking for more info in the first version's IAE
string.
Print a message when an invalid hbase.rootdir is passed
---
Key: HBASE-5327
URL: https://issues.apache.org/jira/browse/HBASE-5327
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jimmy Xiang
Fix For: 0.94.0, 0.90.7, 0.92.1
Attachments: hbase-5327.txt, hbase-5327_v2.txt
As seen on the mailing list:
http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a
path to .oldlogs:
{noformat}
2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
at org.apache.hadoop.fs.Path.initialize(Path.java:148)
at org.apache.hadoop.fs.Path.init(Path.java:71)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
hdfs://sv4r11s38:9100.oldlogs
at java.net.URI.checkPath(URI.java:1787)
at java.net.URI.init(URI.java:735)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
... 6 more
{noformat}
It could also crash anywhere else, this just happens to be the first place we
use hbase.rootdir. We need to verify that it's an actual folder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205089#comment-13205089 ] Jonathan Hsieh commented on HBASE-5364: --- Found an example of debian control files elsewhere where commented license included. Will include. Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch, hbase-5364-0.92.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5363) Automatically run rat check on mvn release builds
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205093#comment-13205093 ] Hadoop QA commented on HBASE-5363: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514046/hbase-5363.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 156 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestHFileBlock Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/933//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/933//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/933//console This message is automatically generated. Automatically run rat check on mvn release builds - Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5363-0.90.patch, hbase-5363.2.patch, hbase-5363.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5373) Table level lock to prevent the race of multiple table level operation
[ https://issues.apache.org/jira/browse/HBASE-5373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205108#comment-13205108 ] Todd Lipcon commented on HBASE-5373: Over in Accumulo land they have a neat framework called FATE (FAult Tolerant Execution or something?) Basically, they put up a znode in ZK for any master-coordinated action, and it acts as an intent log of the idempotent steps. If there's a failover, the new master will finish off any already-running FATE transaction. Table level lock to prevent the race of multiple table level operation -- Key: HBASE-5373 URL: https://issues.apache.org/jira/browse/HBASE-5373 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang A table level lock can guarantee that only one table operation would happen at one time for each table. The master should require and release these table locks correctly during the failover time. One proposal is to keep track of the lock and its corresponding operation in the zookeeper. If there is a master failover, the secondary should have a way to check whether these operations are succeeded nor not before releasing the lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5373) Table level lock to prevent the race of multiple table level operation
[ https://issues.apache.org/jira/browse/HBASE-5373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205120#comment-13205120 ] Liyin Tang commented on HBASE-5373: --- Cool! Sounds like what I try to do here. I will take a look over Accumulo. Table level lock to prevent the race of multiple table level operation -- Key: HBASE-5373 URL: https://issues.apache.org/jira/browse/HBASE-5373 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang A table level lock can guarantee that only one table operation would happen at one time for each table. The master should require and release these table locks correctly during the failover time. One proposal is to keep track of the lock and its corresponding operation in the zookeeper. If there is a master failover, the secondary should have a way to check whether these operations are succeeded nor not before releasing the lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5374) useTableNameGlobally is not initialized for ReaderV2
[
https://issues.apache.org/jira/browse/HBASE-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5374:
--
Affects Version/s: 0.94.0
useTableNameGlobally is not initialized for ReaderV2
Key: HBASE-5374
URL: https://issues.apache.org/jira/browse/HBASE-5374
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Zhihong Yu
SchemaMetrics.useTableNameGlobally is a Boolean object that is not
initialized. It depends on public static method configureGlobally() to
initialize it based on the configuration file. But this is only done for
writer, not for reader. So when invoking hfile tool,
{code}
hbase/bin/hbase org.apache.hadoop.hbase.io.hfile.HFile -v -f YourFile
{code}
where HFileReaderV2 is invoked, it throws exception complaining the flag is
null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize generic arrays not defined previously
[ https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205129#comment-13205129 ] jirapos...@reviews.apache.org commented on HBASE-5358: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3811/#review4999 --- src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java https://reviews.apache.org/r/3811/#comment10996 Ok. Good. - Michael On 2012-02-09 22:18:10, enis wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3811/ bq. --- bq. bq. (Updated 2012-02-09 22:18:10) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). bq. bq. bq. This addresses bug HBASE-5358. bq. https://issues.apache.org/jira/browse/HBASE-5358 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 260f982 bq.src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java 78513ce bq. bq. Diff: https://reviews.apache.org/r/3811/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. enis bq. bq. HBaseObjectWritable should be able to serialize generic arrays not defined previously - Key: HBASE-5358 URL: https://issues.apache.org/jira/browse/HBASE-5358 Project: HBase Issue Type: Improvement Components: coprocessors, io Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HBASE-5358_v3.patch HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205135#comment-13205135 ] Elliott Clark commented on HBASE-5364: -- Thanks Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch, hbase-5364-0.92.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans
[ https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205139#comment-13205139 ] stack commented on HBASE-5325: -- Ok. I can go along w/ cutting the scope of this issue back to its original intent (smile -- sorry for the hairbraining). I was going to make a comment that we have the jmx 'model' not be a completely new one -- that it instead pick the existing model (though admittedly its a bit messy) -- but it looks like your hand has been forced some already. I was going to suggest that we name the mbean for metrics MasterMetrics rather than MasterStatistics but I see this an existing MBean (Who named our mbean 'Master' and 'MasterStatistics' -- our history doesn't say... they don't seem like good names... why not org.apache.hbase prefix... etc.) Why is HBaseMasterMXBean in hbase.metrics and not in hbase.master.metrics? Ditto HbaseRegionServerMXBean You don't need these in your new files: + * Copyright 2012 The Apache Software Foundation Why not name the bean for the master servername especially as there can be multiple masters running in a cluster -- getServerName. Why would you have this info in regionserver jmx attributes: + public String getHBaseMaster(); This is pretty useless: + public long getStartCode(); Better return the regionserver ServerName. Or name the mbean for the ServerName. More to follow... Expose basic information about the master-status through jmx beans --- Key: HBASE-5325 URL: https://issues.apache.org/jira/browse/HBASE-5325 Project: HBase Issue Type: Improvement Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch Similar to the Namenode and Jobtracker, it would be good if the hbase master could expose some information through mbeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HBASE-5199: -- Attachment: (was: HBASE-5199.patch) Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HBASE-5199: -- Attachment: hbase-5199.patch The new patch is rebased on the latest trunk and all the unit tests are passed. Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205163#comment-13205163 ] Hadoop QA commented on HBASE-5199: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514057/hbase-5199.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 156 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplicationPeer org.apache.hadoop.hbase.io.hfile.TestHFileBlock org.apache.hadoop.hbase.TestDrainingServer org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/934//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/934//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/934//console This message is automatically generated. Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4218) Data Block Encoding of KeyValues (aka delta encoding / prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4218: --- Attachment: D1659.2.patch mbautin updated the revision [jira] [HBASE-4218] [89-fb] Porting HFile data block encoding to 89-fb. Reviewers: Kannan, Karthik, nspiegelberg, gqchen, JIRA Fixing DataBlockEncodingTool. Block-level compression parameter was not being handled correctly. REVISION DETAIL https://reviews.facebook.net/D1659 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/KeyValue.java src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java src/main/java/org/apache/hadoop/hbase/client/Result.java src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/CompressionState.java src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.java src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java src/main/java/org/apache/hadoop/hbase/io/encoding/EncoderBufferTooSmallException.java src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCacheKey.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java src/main/java/org/apache/hadoop/hbase/util/ByteBufferUtils.java src/main/java/org/apache/hadoop/hbase/util/CompoundBloomFilter.java src/main/ruby/hbase/admin.rb src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java src/test/java/org/apache/hadoop/hbase/TestKeyValue.java src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java src/test/java/org/apache/hadoop/hbase/io/encoding/RedundantKVGenerator.java src/test/java/org/apache/hadoop/hbase/io/encoding/TestBufferedDataBlockEncoder.java src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java src/test/java/org/apache/hadoop/hbase/io/encoding/TestEncodedSeekers.java src/test/java/org/apache/hadoop/hbase/io/encoding/TestUpgradeFromHFileV1ToEncoding.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205173#comment-13205173 ] Phabricator commented on HBASE-5199: mbautin has commented on the revision [jira][HBASE-5199] Delete out of TTL store files before compaction selection . @Liyin: could you please post the rebased version of the patch so I can commit? REVISION DETAIL https://reviews.facebook.net/D1311 Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205174#comment-13205174 ] Phabricator commented on HBASE-5199: mbautin has commented on the revision [jira][HBASE-5199] Delete out of TTL store files before compaction selection . @Liyin: could you please post the rebased version of the patch so I can commit? REVISION DETAIL https://reviews.facebook.net/D1311 Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205172#comment-13205172 ] Phabricator commented on HBASE-5199: mbautin has commented on the revision [jira][HBASE-5199] Delete out of TTL store files before compaction selection . @Liyin: could you please post the rebased version of the patch so I can commit? REVISION DETAIL https://reviews.facebook.net/D1311 Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205175#comment-13205175 ] Phabricator commented on HBASE-5199: mbautin has commented on the revision [jira][HBASE-5199] Delete out of TTL store files before compaction selection . @Liyin: could you please post the rebased version of the patch so I can commit? REVISION DETAIL https://reviews.facebook.net/D1311 Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5199) Delete out of TTL store files before compaction selection
[ https://issues.apache.org/jira/browse/HBASE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205171#comment-13205171 ] Phabricator commented on HBASE-5199: mbautin has commented on the revision [jira][HBASE-5199] Delete out of TTL store files before compaction selection . @Liyin: could you please post the rebased version of the patch so I can commit? REVISION DETAIL https://reviews.facebook.net/D1311 Delete out of TTL store files before compaction selection - Key: HBASE-5199 URL: https://issues.apache.org/jira/browse/HBASE-5199 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Attachments: D1311.1.patch, D1311.2.patch, D1311.3.patch, D1311.4.patch, D1311.5.patch, D1311.5.patch, hbase-5199.patch Currently, HBase deletes the out of TTL store files after compaction. We can change the sequence to delete the out of TTL store files before selecting store files for compactions. In this way, HBase can keep deleting the old invalid store files without compaction, and also prevent from unnecessary compactions since the out of TTL store files will be deleted before the compaction selection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4218) Data Block Encoding of KeyValues (aka delta encoding / prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205176#comment-13205176 ] Hadoop QA commented on HBASE-4218: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514065/D1659.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 106 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/935//console This message is automatically generated. Data Block Encoding of KeyValues (aka delta encoding / prefix compression) --- Key: HBASE-4218 URL: https://issues.apache.org/jira/browse/HBASE-4218 Project: HBase Issue Type: Improvement Components: io Affects Versions: 0.94.0 Reporter: Jacek Migdal Assignee: Mikhail Bautin Labels: compression Fix For: 0.94.0 Attachments: 0001-Delta-encoding-fixed-encoded-scanners.patch, 0001-Delta-encoding.patch, 4218-2012-01-14.txt, 4218-v16.txt, 4218.txt, D1659.1.patch, D1659.2.patch, D447.1.patch, D447.10.patch, D447.11.patch, D447.12.patch, D447.13.patch, D447.14.patch, D447.15.patch, D447.16.patch, D447.17.patch, D447.18.patch, D447.19.patch, D447.2.patch, D447.20.patch, D447.21.patch, D447.22.patch, D447.23.patch, D447.24.patch, D447.25.patch, D447.26.patch, D447.3.patch, D447.4.patch, D447.5.patch, D447.6.patch, D447.7.patch, D447.8.patch, D447.9.patch, Data-block-encoding-2011-12-23.patch, Delta-encoding-2012-01-17_11_09_09.patch, Delta-encoding-2012-01-25_00_45_29.patch, Delta-encoding-2012-01-25_16_32_14.patch, Delta-encoding.patch-2011-12-22_11_52_07.patch, Delta-encoding.patch-2012-01-05_15_16_43.patch, Delta-encoding.patch-2012-01-05_16_31_44.patch, Delta-encoding.patch-2012-01-05_16_31_44_copy.patch, Delta-encoding.patch-2012-01-05_18_50_47.patch, Delta-encoding.patch-2012-01-07_14_12_48.patch, Delta-encoding.patch-2012-01-13_12_20_07.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff A compression for keys. Keys are sorted in HFile and they are usually very similar. Because of that, it is possible to design better compression than general purpose algorithms, It is an additional step designed to be used in memory. It aims to save memory in cache as well as speeding seeks within HFileBlocks. It should improve performance a lot, if key lengths are larger than value lengths. For example, it makes a lot of sense to use it when value is a counter. Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) shows that I could achieve decent level of compression: key compression ratio: 92% total compression ratio: 85% LZO on the same data: 85% LZO after delta encoding: 91% While having much better performance (20-80% faster decompression ratio than LZO). Moreover, it should allow far more efficient seeking which should improve performance a bit. It seems that a simple compression algorithms are good enough. Most of the savings are due to prefix compression, int128 encoding, timestamp diffs and bitfields to avoid duplication. That way, comparisons of compressed data can be much faster than a byte comparator (thanks to prefix compression and bitfields). In order to implement it in HBase two important changes in design will be needed: -solidify interface to HFileBlock / HFileReader Scanner to provide seeking and iterating; access to uncompressed buffer in HFileBlock will have bad performance -extend comparators to support comparison assuming that N first bytes are equal (or some fields are equal) Link to a discussion about something similar: http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windowssubj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4516) HFile-level load tester with compaction and random-read workloads
[ https://issues.apache.org/jira/browse/HBASE-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin resolved HBASE-4516. --- Resolution: Fixed Resolved as part of HBASE-4218. HFile-level load tester with compaction and random-read workloads - Key: HBASE-4516 URL: https://issues.apache.org/jira/browse/HBASE-4516 Project: HBase Issue Type: Test Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Fix For: 0.94.0 This is a load testing tool for HFile implementations, which supports two workloads: - Compactions (merge the input HFiles). A special case of this is only one input, which allows to do HFile format conversions. - Random reads. Launches the specified number of threads that do seeks and short scans on randomly generated keys. The original purpose of this tool was to ensure that HFile format v2 did not introduce performance regressions. Keys for the read workload are generated randomly between the first and the last key of the HFile. At each position, instead of precisely calculating the correct probability for every byte value b, we select a uniformly random byte between in the allowed [low, high] range. In addition, there is a heuristic that determines the positions at which the key has hex characters, and the random key contains hex characters at those positions as well. Example output for the random read workload: Time: 120 sec, seek/sec: 8290, kv/sec: 30351, kv bytes/sec: 91868121, blk/sec: 10147, unique keys: 232779 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5230) Ensure compactions do not cache-on-write data blocks
[ https://issues.apache.org/jira/browse/HBASE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-5230: -- Resolution: Fixed Release Note: Committed into both trunk and 89-fb Status: Resolved (was: Patch Available) Ensure compactions do not cache-on-write data blocks Key: HBASE-5230 URL: https://issues.apache.org/jira/browse/HBASE-5230 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D1353.1.patch, D1353.2.patch, D1353.3.patch, D1353.4.patch, Don-t-cache-data-blocks-on-compaction-2012-01-21_00_53_54.patch, Don-t-cache-data-blocks-on-compaction-2012-01-23_10_23_45.patch, Don-t-cache-data-blocks-on-compaction-2012-01-23_15_27_23.patch Create a unit test for HBASE-3976 (making sure we don't cache data blocks on write during compactions even if cache-on-write is enabled generally enabled). This is because we have very different implementations of HBASE-3976 without HBASE-4422 CacheConfig (on top of 89-fb, created by Liyin) and with CacheConfig (presumably it's there but not sure if it even works, since the patch in HBASE-3976 may not have been committed). We need to create a unit test to verify that we don't cache data blocks on write during compactions, and resolve HBASE-3976 so that this new unit test does not fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5230) Ensure compactions do not cache-on-write data blocks
[ https://issues.apache.org/jira/browse/HBASE-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-5230: -- Release Note: (was: Committed into both trunk and 89-fb) Ensure compactions do not cache-on-write data blocks Key: HBASE-5230 URL: https://issues.apache.org/jira/browse/HBASE-5230 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D1353.1.patch, D1353.2.patch, D1353.3.patch, D1353.4.patch, Don-t-cache-data-blocks-on-compaction-2012-01-21_00_53_54.patch, Don-t-cache-data-blocks-on-compaction-2012-01-23_10_23_45.patch, Don-t-cache-data-blocks-on-compaction-2012-01-23_15_27_23.patch Create a unit test for HBASE-3976 (making sure we don't cache data blocks on write during compactions even if cache-on-write is enabled generally enabled). This is because we have very different implementations of HBASE-3976 without HBASE-4422 CacheConfig (on top of 89-fb, created by Liyin) and with CacheConfig (presumably it's there but not sure if it even works, since the patch in HBASE-3976 may not have been committed). We need to create a unit test to verify that we don't cache data blocks on write during compactions, and resolve HBASE-3976 so that this new unit test does not fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5368) Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible in HBase installs
[ https://issues.apache.org/jira/browse/HBASE-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5368: -- Status: Patch Available (was: Open) Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible in HBase installs - Key: HBASE-5368 URL: https://issues.apache.org/jira/browse/HBASE-5368 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5368.txt Very simple change to make PrefixSplitKeyPolicy accessible in HBase installs (user still needs to setup the table(s) accordingly). Right now it is in src/test/org.apache.hadoop.hbase.regionserver, I propose moving it to src/org.apache.hadoop.hbase.regionserver (alongside ConstantSizeRegionSplitPolicy), and maybe renaming it too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays
[ https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5358: -- Fix Version/s: 0.94.0 Hadoop Flags: Reviewed Summary: HBaseObjectWritable should be able to serialize/deserialize generic arrays (was: HBaseObjectWritable should be able to serialize generic arrays not defined previously) HBaseObjectWritable should be able to serialize/deserialize generic arrays -- Key: HBASE-5358 URL: https://issues.apache.org/jira/browse/HBASE-5358 Project: HBase Issue Type: Improvement Components: coprocessors, io Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.94.0 Attachments: HBASE-5358_v3.patch HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays
[ https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205196#comment-13205196 ] Zhihong Yu commented on HBASE-5358: --- Integrated to TRUNK. Thanks for the patch Enis. Thanks for the review Michael. HBaseObjectWritable should be able to serialize/deserialize generic arrays -- Key: HBASE-5358 URL: https://issues.apache.org/jira/browse/HBASE-5358 Project: HBase Issue Type: Improvement Components: coprocessors, io Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.94.0 Attachments: HBASE-5358_v3.patch HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4683) Always cache index and bloom blocks
[ https://issues.apache.org/jira/browse/HBASE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205200#comment-13205200 ] Mikhail Bautin commented on HBASE-4683: --- @jdcryans: it looks like TestForceCacheImportantBlocks.java was not included into your trunk commit of this JIRA. I will verify that the test still works and post an addendum patch. Always cache index and bloom blocks --- Key: HBASE-4683 URL: https://issues.apache.org/jira/browse/HBASE-4683 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Mikhail Bautin Priority: Minor Fix For: 0.94.0, 0.92.0 Attachments: 0001-Cache-important-block-types.patch, 4683-v2.txt, 4683.txt, D807.1.patch, D807.2.patch, D807.3.patch, HBASE-4683-0.92-v2.patch, HBASE-4683-v3.patch This would add a new boolean config option: hfile.block.cache.datablocks Default would be true. Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the (aggregate) cache. This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets). I would like to get a general feeling about what folks think about this. The change itself would be simple. Update (Mikhail): we probably don't need a new conf option. Instead, we will make index blocks cached by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5368) Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible in HBase installs
[ https://issues.apache.org/jira/browse/HBASE-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205204#comment-13205204 ] Hadoop QA commented on HBASE-5368: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514067/5368.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 157 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/936//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/936//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/936//console This message is automatically generated. Move PrefixSplitKeyPolicy out of the src/test into src, so it is accessible in HBase installs - Key: HBASE-5368 URL: https://issues.apache.org/jira/browse/HBASE-5368 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5368.txt Very simple change to make PrefixSplitKeyPolicy accessible in HBase installs (user still needs to setup the table(s) accordingly). Right now it is in src/test/org.apache.hadoop.hbase.regionserver, I propose moving it to src/org.apache.hadoop.hbase.regionserver (alongside ConstantSizeRegionSplitPolicy), and maybe renaming it too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5327) Print a message when an invalid hbase.rootdir is passed
[
https://issues.apache.org/jira/browse/HBASE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205210#comment-13205210
]
Jonathan Hsieh commented on HBASE-5327:
---
Jimmy and I were chatting about the funny control flow and found it strange
that {{new Path(invalidRootDir)}} didn't through IllegalArgumentException,
while {{ new Path (new Path(invalidRootDir), .olddirs)}} threw an
IllegalArguentException. (I was expecting the mkdirs to IAE, not exists).
This explains why the slightly odd control flow is necessary and makes me much
happier with v2 patch.
Maybe someone with more HDFS background can chime in about why this seeming
inconsistency exists?
Print a message when an invalid hbase.rootdir is passed
---
Key: HBASE-5327
URL: https://issues.apache.org/jira/browse/HBASE-5327
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jimmy Xiang
Fix For: 0.94.0, 0.90.7, 0.92.1
Attachments: hbase-5327.txt, hbase-5327_v2.txt
As seen on the mailing list:
http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24124
If hbase.rootdir doesn't specify a folder on hdfs we crash while opening a
path to .oldlogs:
{noformat}
2012-02-02 23:07:26,292 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
path in absolute URI: hdfs://sv4r11s38:9100.oldlogs
at org.apache.hadoop.fs.Path.initialize(Path.java:148)
at org.apache.hadoop.fs.Path.init(Path.java:71)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:448)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.URISyntaxException: Relative path in absolute URI:
hdfs://sv4r11s38:9100.oldlogs
at java.net.URI.checkPath(URI.java:1787)
at java.net.URI.init(URI.java:735)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
... 6 more
{noformat}
It could also crash anywhere else, this just happens to be the first place we
use hbase.rootdir. We need to verify that it's an actual folder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5312) Closed parent region present in Hlog.lastSeqWritten
[
https://issues.apache.org/jira/browse/HBASE-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205214#comment-13205214
]
Jimmy Xiang commented on HBASE-5312:
Have anyone seen this issue on 0.92 release? Could we add some logging so that
we will have some clue when it happens again?
Closed parent region present in Hlog.lastSeqWritten
---
Key: HBASE-5312
URL: https://issues.apache.org/jira/browse/HBASE-5312
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Fix For: 0.90.7
This is in reference to the mail sent in the dev mailing list
Closed parent region present in Hlog.lastSeqWritten.
The sceanrio described is
We had a region that was split into two daughters. When the hlog roll tried
to flush the region there was an entry in the HLog.lastSeqWritten that was
not flushed or removed from the lastSeqWritten during the parent close.
Because this flush was not happening subsequent flushes were getting blocked
{code}
05:06:44,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
hlogs: logs=122, maxlogs=32; forcing flush of 1 regions(s):
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:06:44,422 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
requester=null
05:10:48,666 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
hlogs: logs=123, maxlogs=32; forcing flush of 1 regions(s):
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:10:48,666 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
requester=null
05:14:46,075 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
hlogs: logs=124, maxlogs=32; forcing flush of 1 regions(s):
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:14:46,075 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
requester=null
05:15:41,584 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
hlogs: logs=125, maxlogs=32; forcing flush of 1 regions(s):
2acaf8e3acfd2e8a5825a1f6f0aca4a8
05:15:41,584 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
{code}
Lets see what happened for the region 2acaf8e3acfd2e8a5825a1f6f0aca4a8
{code}
2012-01-06 00:30:55,214 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming flushed file at
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/.tmp/1755862026714756815
to
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123
2012-01-06 00:30:58,946 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Instantiated
Htable_UFDR_016,049790700093168-0456520,1325809837958.0ebe5bd7fcbc09ee074d5600b9d4e062.
2012-01-06 00:30:59,614 INFO org.apache.hadoop.hbase.regionserver.Store:
Added
hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123,
entries=7537, sequenceid=20312223, memsize=4.2m, filesize=2.9m
2012-01-06 00:30:59,787 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2012-01-06 00:30:59,787 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~133.5m for region
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in
21816ms, sequenceid=20312223, compaction requested=true
2012-01-06 00:30:59,787 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
because regionserver20020.cacheFlusher; priority=0, compaction queue size=5840
{code}
A user triggered split has been issued to this region which can be seen in
the above logs.
The flushing of this region has resulted in a seq id 20312223.
The region has been splitted and the parent region has been closed
{code}
00:31:12,607 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction:
Starting split of region
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.:
disabling compactions flushes
00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates
disabled for region
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
00:31:13,718 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
00:31:39,552 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Offlined
[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans
[ https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205215#comment-13205215 ] stack commented on HBASE-5325: -- I suppose you can't name the bean for the ServerName because you need to be able to locate the bean -- i.e. there'll be N instances of RegionServer beans and you'd then figure which belonged to which by looking at the ServerName attribute (I remember this is how it worked now). Is this in branch-0.20-append branch: +import org.apache.hadoop.metrics2.util.MBeans; If not, this breaks our ability to run on that branch (Up to this presumed we could run there). Its in 1.0.0 hadoop? (would need to check its in CDH..) Do classes have to have an HBase (or Hbase) prefix? Seems redundant (and we should be consistent). Can we have a better name than HBaseRegionServerInfo. It says nothing. If it was RegionServerMBean or MXBean, it'd say more about what this class is about. You can all it 'instance'. 'theInstance' is too much? And again, publishing master, ensemble, and startcode seems like a bunch of info you'd never act on. Master maybe -- you'd know which master it was talking too -- and perhaps ensemble because then you know who its registered with (though having both seems unnessary... the ensemble with its rootdir will tell you which cluster we belong too... perhaps you should get the cluster id out here?) but startcode is no good to anyone really. Should be ServerName coming out here. Thats how we uniquely identify regionservers in fs, when they report into the master and up in ensemble. Might as well continue the identifier here. Does this class need to take a RegionServer implementation? Can it take a o.a.h.h.Server and/or a o.a.h.h.regionserver.RegionServerServices publishing jmx attributes? These are Interfaces. Might make this all easier to write tests on. HbaseRegionServerMXBean should be in the regionserver/metrics package then you could call it MXBean or ServerMXBean. HBaseMasterMXBean should be min master/metrics, should be HBaseMasterMXBean. The RegionServerInfo in it should be showing more than this small set of metrics if you are going to the bother of putting this stuff out on jmx (we do requests per region now -- should there be one of these classes per region?) I think that you don't need startcode if getRegionServer is returning the ServerName as a String. Don't RegionState up in RegionsInTransiation have a ServerName associated too? YOu're not publishing this? Again, ain't these bad names for beans up in jmx? +mxBean = MBeans.register(HBase, MasterInfo, +mxBeanInfo); shouldn't it be org.apache.hbase... or hbase if thats the parent for all of the hbase beans (Ain't there convention on bean naming -- IIRC). MasterInfo says nothing. Could it be just Master. Then you do getServerName or what ever it is that returns ServerName to distingush this master from the backup Master? Sorry for so many comments for such a small patch. I just feel that this stuff can be really useful if its done right. Else its just more stuff for us to maintain. Thanks for doing this stuff lads. Expose basic information about the master-status through jmx beans --- Key: HBASE-5325 URL: https://issues.apache.org/jira/browse/HBASE-5325 Project: HBase Issue Type: Improvement Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch Similar to the Namenode and Jobtracker, it would be good if the hbase master could expose some information through mbeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4683) Always cache index and bloom blocks
[ https://issues.apache.org/jira/browse/HBASE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4683: --- Attachment: D1695.1.patch mbautin requested code review of [jira] [HBASE-4683] Test that we always cache index and bloom blocks. Reviewers: JIRA, jdcryans, lhofhansl, Liyin This was reviewed D807 but was not checked in. Submitting unit test as a separate patch, and extending the scope of the test to also handle the case when block cache is enabled for the column family. TEST PLAN Run unit tests REVISION DETAIL https://reviews.facebook.net/D1695 AFFECTED FILES src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3621/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Always cache index and bloom blocks --- Key: HBASE-4683 URL: https://issues.apache.org/jira/browse/HBASE-4683 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Mikhail Bautin Priority: Minor Fix For: 0.94.0, 0.92.0 Attachments: 0001-Cache-important-block-types.patch, 4683-v2.txt, 4683.txt, D1695.1.patch, D807.1.patch, D807.2.patch, D807.3.patch, HBASE-4683-0.92-v2.patch, HBASE-4683-v3.patch This would add a new boolean config option: hfile.block.cache.datablocks Default would be true. Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the (aggregate) cache. This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets). I would like to get a general feeling about what folks think about this. The change itself would be simple. Update (Mikhail): we probably don't need a new conf option. Instead, we will make index blocks cached by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5375) Ensure that compactions use already cached blocks but do not cache new data blocks
Ensure that compactions use already cached blocks but do not cache new data blocks -- Key: HBASE-5375 URL: https://issues.apache.org/jira/browse/HBASE-5375 Project: HBase Issue Type: Test Reporter: Mikhail Bautin Create a unit test to verify that compactions reuse existing cached blocks but do not thrash the cache with newly read blocks. Also need to verify that we only read every data block once, e.g. that we don't re-read the block on every next() operation. HBASE-1597 did not seem to include a unit test, so we need to add a test now. This and HBASE-4683 (the unit test that was not checked in) are the remaining missing pieces before we can close HBASE-3976. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5376) Add more logging to triage HBASE-5312: Closed parent region present in Hlog.lastSeqWritten
Add more logging to triage HBASE-5312: Closed parent region present in Hlog.lastSeqWritten -- Key: HBASE-5376 URL: https://issues.apache.org/jira/browse/HBASE-5376 Project: HBase Issue Type: Sub-task Reporter: Jimmy Xiang Priority: Minor It is hard to find out what exactly caused HBASE-5312. Some logging will be helpful to shine some lights. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5377) Fix licenses on the 0.90 branch.
[ https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205225#comment-13205225 ] Jonathan Hsieh commented on HBASE-5377: --- This is partially a backport of HBASE-4647 but also fixes other files that are only relevant on the 0.90 branch. Fix licenses on the 0.90 branch. Key: HBASE-5377 URL: https://issues.apache.org/jira/browse/HBASE-5377 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: Jonathan Hsieh There are a handful of empty files and several files missing apache licenses on the 0.90 branch. This patch will fixes all of them and in conjunction with HBASE-5363 will allow it to pass RAT tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5377) Fix licenses on the 0.90 branch.
[ https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5377: -- Attachment: hbase-5377.patch Fix licenses on the 0.90 branch. Key: HBASE-5377 URL: https://issues.apache.org/jira/browse/HBASE-5377 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: Jonathan Hsieh Attachments: hbase-5377.patch There are a handful of empty files and several files missing apache licenses on the 0.90 branch. This patch will fixes all of them and in conjunction with HBASE-5363 will allow it to pass RAT tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5377) Fix licenses on the 0.90 branch.
[ https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-5377: - Assignee: Jonathan Hsieh Fix licenses on the 0.90 branch. Key: HBASE-5377 URL: https://issues.apache.org/jira/browse/HBASE-5377 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5377.patch There are a handful of empty files and several files missing apache licenses on the 0.90 branch. This patch will fixes all of them and in conjunction with HBASE-5363 will allow it to pass RAT tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5377) Fix licenses on the 0.90 branch.
[ https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5377: -- Status: Patch Available (was: Open) Fix licenses on the 0.90 branch. Key: HBASE-5377 URL: https://issues.apache.org/jira/browse/HBASE-5377 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: Jonathan Hsieh Attachments: hbase-5377.patch There are a handful of empty files and several files missing apache licenses on the 0.90 branch. This patch will fixes all of them and in conjunction with HBASE-5363 will allow it to pass RAT tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans
[ https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205234#comment-13205234 ] Hitesh Shah commented on HBASE-5325: @Stack, thanks for the comments. Will go over them and submit a modified patch soon. Expose basic information about the master-status through jmx beans --- Key: HBASE-5325 URL: https://issues.apache.org/jira/browse/HBASE-5325 Project: HBase Issue Type: Improvement Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch Similar to the Namenode and Jobtracker, it would be good if the hbase master could expose some information through mbeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5377) Fix licenses on the 0.90 branch.
[
https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205240#comment-13205240
]
Jonathan Hsieh commented on HBASE-5377:
---
server info web pages look good.
'mvn site' to build docs seem to have a problem, but this is present before and
after.
{code}
...
[INFO] --- maven-site-plugin:2.0.1:site (default-site) @ hbase ---
[INFO] Unable to load parent project from a relative path: 1 problem was
encountered while building the effective model
[FATAL] Non-readable POM /home/jon/proj/pom.xml: /home/jon/proj/pom.xml (No
such file or directory) @
for project at /home/jon/proj/pom.xml for project at /home/jon/proj/pom.xml
[INFO] Parent project loaded from repository.
[INFO] Unable to load parent project from a relative path: 1 problem was
encountered while building the effective model
[FATAL] Non-readable POM /home/jon/proj/pom.xml: /home/jon/proj/pom.xml (No
such file or directory) @
for project at /home/jon/proj/pom.xml for project at /home/jon/proj/pom.xml
[INFO] Parent project loaded from repository.
[INFO]
[INFO] BUILD SUCCESS
[INFO]
...
{code}
Fix licenses on the 0.90 branch.
Key: HBASE-5377
URL: https://issues.apache.org/jira/browse/HBASE-5377
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5377.patch
There are a handful of empty files and several files missing apache licenses
on the 0.90 branch. This patch will fixes all of them and in conjunction
with HBASE-5363 will allow it to pass RAT tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5377) Fix licenses on the 0.90 branch.
[ https://issues.apache.org/jira/browse/HBASE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205239#comment-13205239 ] Hadoop QA commented on HBASE-5377: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514077/hbase-5377.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/937//console This message is automatically generated. Fix licenses on the 0.90 branch. Key: HBASE-5377 URL: https://issues.apache.org/jira/browse/HBASE-5377 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5377.patch There are a handful of empty files and several files missing apache licenses on the 0.90 branch. This patch will fixes all of them and in conjunction with HBASE-5363 will allow it to pass RAT tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5376) Add more logging to triage HBASE-5312: Closed parent region present in Hlog.lastSeqWritten
[ https://issues.apache.org/jira/browse/HBASE-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205250#comment-13205250 ] Todd Lipcon commented on HBASE-5376: I bet we could also write a stress test - sounds like some race where we don't block off updates at the right spot when closing a region. Maybe a test which continually opens/closes a region while making edits, and after every close, verify it's not in the map? Add more logging to triage HBASE-5312: Closed parent region present in Hlog.lastSeqWritten -- Key: HBASE-5376 URL: https://issues.apache.org/jira/browse/HBASE-5376 Project: HBase Issue Type: Sub-task Reporter: Jimmy Xiang Priority: Minor Fix For: 0.90.7 It is hard to find out what exactly caused HBASE-5312. Some logging will be helpful to shine some lights. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5075) regionserver crashed and failover
[ https://issues.apache.org/jira/browse/HBASE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 代志远 updated HBASE-5075: --- Fix Version/s: 0.90.5 regionserver crashed and failover - Key: HBASE-5075 URL: https://issues.apache.org/jira/browse/HBASE-5075 Project: HBase Issue Type: Improvement Components: monitoring, regionserver, replication, zookeeper Affects Versions: 0.92.1 Reporter: 代志远 Fix For: 0.90.5 regionserver crashed,it is too long time to notify hmaster.when hmaster know regionserver's shutdown,it is long time to fetch the hlog's lease. hbase is a online db, availability is very important. i have a idea to improve availability, monitor node to check regionserver's pid.if this pid not exsits,i think the rs down,i will delete the znode,and force close the hlog file. so the period maybe 100ms. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans
[ https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205251#comment-13205251 ] Todd Lipcon commented on HBASE-5325: No, metrics2 is not in CDH, since it's neither compatible with earlier versions of metrics1 nor is it compatible with the metrics2 in 0.23+. Expose basic information about the master-status through jmx beans --- Key: HBASE-5325 URL: https://issues.apache.org/jira/browse/HBASE-5325 Project: HBase Issue Type: Improvement Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch Similar to the Namenode and Jobtracker, it would be good if the hbase master could expose some information through mbeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays
[ https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205255#comment-13205255 ] Hudson commented on HBASE-5358: --- Integrated in HBase-TRUNK-security #104 (See [https://builds.apache.org/job/HBase-TRUNK-security/104/]) HBASE-5358 HBaseObjectWritable should be able to serialize/deserialize generic arrays tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java HBaseObjectWritable should be able to serialize/deserialize generic arrays -- Key: HBASE-5358 URL: https://issues.apache.org/jira/browse/HBASE-5358 Project: HBase Issue Type: Improvement Components: coprocessors, io Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.94.0 Attachments: HBASE-5358_v3.patch HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5075) regionserver crashed and failover
[ https://issues.apache.org/jira/browse/HBASE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205258#comment-13205258 ] Zhihong Yu commented on HBASE-5075: --- One solution is to expire region server znode more quickly. regionserver crashed and failover - Key: HBASE-5075 URL: https://issues.apache.org/jira/browse/HBASE-5075 Project: HBase Issue Type: Improvement Components: monitoring, regionserver, replication, zookeeper Affects Versions: 0.92.1 Reporter: 代志远 Fix For: 0.90.5 regionserver crashed,it is too long time to notify hmaster.when hmaster know regionserver's shutdown,it is long time to fetch the hlog's lease. hbase is a online db, availability is very important. i have a idea to improve availability, monitor node to check regionserver's pid.if this pid not exsits,i think the rs down,i will delete the znode,and force close the hlog file. so the period maybe 100ms. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans
[ https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205260#comment-13205260 ] Hitesh Shah commented on HBASE-5325: bq. No, metrics2 is not in CDH, since it's neither compatible with earlier versions of metrics1 nor is it compatible with the metrics2 in 0.23+. Will make the change to use metrics.MBeanUtil#registerMBean in that case. I am assuming that will meet all branches' needs? Expose basic information about the master-status through jmx beans --- Key: HBASE-5325 URL: https://issues.apache.org/jira/browse/HBASE-5325 Project: HBase Issue Type: Improvement Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5325.1.patch, HBASE-5325.wip.patch Similar to the Namenode and Jobtracker, it would be good if the hbase master could expose some information through mbeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup
[ https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5209: - Fix Version/s: 0.92.1 0.94.0 Status: Patch Available (was: Open) I tested this by: * Running mvn -P localTests -Dtest=TestMasterFailover test successfully * Starting a master and a couple of backup masters, checking zk_dump, killing the master, checking zk_dump, restarting the master, checking zk_dump. All checks displayed expected values in the master and backupmaster ZNodes. HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup Key: HBASE-5209 URL: https://issues.apache.org/jira/browse/HBASE-5209 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.92.0, 0.90.5, 0.94.0 Reporter: Aditya Acharya Assignee: David S. Wang Fix For: 0.94.0, 0.92.1 Attachments: HBASE-5209-v0.diff I have a multi-master HBase set up, and I'm trying to programmatically determine which of the masters is currently active. But the API does not allow me to do this. There is a getMaster() method in the HConnection class, but it returns an HMasterInterface, whose methods do not allow me to find out which master won the last race. The API should have a getActiveMasterHostname() or something to that effect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup
[ https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5209: - Attachment: HBASE-5209-v0.diff HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup Key: HBASE-5209 URL: https://issues.apache.org/jira/browse/HBASE-5209 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Aditya Acharya Assignee: David S. Wang Fix For: 0.94.0, 0.92.1 Attachments: HBASE-5209-v0.diff I have a multi-master HBase set up, and I'm trying to programmatically determine which of the masters is currently active. But the API does not allow me to do this. There is a getMaster() method in the HConnection class, but it returns an HMasterInterface, whose methods do not allow me to find out which master won the last race. The API should have a getActiveMasterHostname() or something to that effect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup
[ https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205269#comment-13205269 ] Hadoop QA commented on HBASE-5209: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514082/HBASE-5209-v0.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/938//console This message is automatically generated. HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup Key: HBASE-5209 URL: https://issues.apache.org/jira/browse/HBASE-5209 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Aditya Acharya Assignee: David S. Wang Fix For: 0.94.0, 0.92.1 Attachments: HBASE-5209-v0.diff I have a multi-master HBase set up, and I'm trying to programmatically determine which of the masters is currently active. But the API does not allow me to do this. There is a getMaster() method in the HConnection class, but it returns an HMasterInterface, whose methods do not allow me to find out which master won the last race. The API should have a getActiveMasterHostname() or something to that effect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup
[ https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205270#comment-13205270 ] David S. Wang commented on HBASE-5209: -- I also ran hbase hbck -details successfully in order to exercise the changes in HBaseFsck.java. HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup Key: HBASE-5209 URL: https://issues.apache.org/jira/browse/HBASE-5209 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Aditya Acharya Assignee: David S. Wang Fix For: 0.94.0, 0.92.1 Attachments: HBASE-5209-v0.diff I have a multi-master HBase set up, and I'm trying to programmatically determine which of the masters is currently active. But the API does not allow me to do this. There is a getMaster() method in the HConnection class, but it returns an HMasterInterface, whose methods do not allow me to find out which master won the last race. The API should have a getActiveMasterHostname() or something to that effect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup
[ https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5209: - Status: Open (was: Patch Available) Need to resubmit new patch with correct formatting. HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup Key: HBASE-5209 URL: https://issues.apache.org/jira/browse/HBASE-5209 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.92.0, 0.90.5, 0.94.0 Reporter: Aditya Acharya Assignee: David S. Wang Fix For: 0.94.0, 0.92.1 Attachments: HBASE-5209-v0.diff, HBASE-5209-v1.diff I have a multi-master HBase set up, and I'm trying to programmatically determine which of the masters is currently active. But the API does not allow me to do this. There is a getMaster() method in the HConnection class, but it returns an HMasterInterface, whose methods do not allow me to find out which master won the last race. The API should have a getActiveMasterHostname() or something to that effect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup
[ https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5209: - Fix Version/s: 0.90.7 HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup Key: HBASE-5209 URL: https://issues.apache.org/jira/browse/HBASE-5209 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Aditya Acharya Assignee: David S. Wang Fix For: 0.94.0, 0.90.7, 0.92.1 Attachments: HBASE-5209-v0.diff, HBASE-5209-v1.diff I have a multi-master HBase set up, and I'm trying to programmatically determine which of the masters is currently active. But the API does not allow me to do this. There is a getMaster() method in the HConnection class, but it returns an HMasterInterface, whose methods do not allow me to find out which master won the last race. The API should have a getActiveMasterHostname() or something to that effect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API
[ https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205281#comment-13205281 ] jirapos...@reviews.apache.org commented on HBASE-5371: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3829/#review5003 --- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java https://reviews.apache.org/r/3829/#comment11004 This seems a reasonable approach. security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java https://reviews.apache.org/r/3829/#comment11005 Perhaps add a couple of additional cases to check? security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java https://reviews.apache.org/r/3829/#comment11006 Likewise - Andrew On 2012-02-09 23:12:00, enis wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/3829/ bq. --- bq. bq. (Updated 2012-02-09 23:12:00) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. We need to introduce something like AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so that clients can check access rights before carrying out the operations. We need this kind of operation for HCATALOG-245, which introduces authorization providers for hbase over hcat. We cannot use getUserPermissions() since it requires ADMIN permissions on the global/table level. bq. bq. bq. This addresses bug HBASE-5371. bq. https://issues.apache.org/jira/browse/HBASE-5371 bq. bq. bq. Diffs bq. - bq. bq. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java 5091b7d bq. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java 5fa2edb bq. security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java f864373 bq. bq. Diff: https://reviews.apache.org/r/3829/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. enis bq. bq. Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API Key: HBASE-5371 URL: https://issues.apache.org/jira/browse/HBASE-5371 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0, 0.92.1 Reporter: Enis Soztutar Assignee: Enis Soztutar We need to introduce something like AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so that clients can check access rights before carrying out the operations. We need this kind of operation for HCATALOG-245, which introduces authorization providers for hbase over hcat. We cannot use getUserPermissions() since it requires ADMIN permissions on the global/table level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5372) Table mutation operations should check table level rights, not global rights
[ https://issues.apache.org/jira/browse/HBASE-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205282#comment-13205282 ] Andrew Purtell commented on HBASE-5372: --- bq. However, for this maybe we have to revisit table ownership rights. Currently, the table owner has every right on the table, and this is not managed through the normal grant/revoke operations, but on the table metadata. We may want to remove table ownership, but introduce default table creation rights, which means that when a user creates a table, she automatically get those rights allocated. But another user can grant extra rights, or revoke them. Sure, makes sense. We opted for simplicity in the initial implementation. Table mutation operations should check table level rights, not global rights - Key: HBASE-5372 URL: https://issues.apache.org/jira/browse/HBASE-5372 Project: HBase Issue Type: Sub-task Components: security Reporter: Enis Soztutar Assignee: Enis Soztutar getUserPermissions(tableName)/grant/revoke and drop/modify table operations should not check for global CREATE/ADMIN rights, but table CREATE/ADMIN rights. The reasoning is that if a user is able to admin or read from a table, she should be able to read the table's permissions. We can choose whether we want only READ or ADMIN permissions for getUserPermission(). Since we check for global permissions first for table permissions, configuring table access using global permissions will continue to work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
