[jira] [Commented] (HBASE-10201) Port 'Make flush decisions per column family' to trunk
[
https://issues.apache.org/jira/browse/HBASE-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245245#comment-14245245
]
Hadoop QA commented on HBASE-10201:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687023/HBASE-10201_19.patch
against master branch at commit a0e473730e2cd819e7442dbd2b332d7833755ba2.
ATTACHMENT ID: 12687023
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 32 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12065//console
This message is automatically generated.
Port 'Make flush decisions per column family' to trunk
--
Key: HBASE-10201
URL: https://issues.apache.org/jira/browse/HBASE-10201
Project: HBase
Issue Type: Improvement
Components: wal
Reporter: Ted Yu
Assignee: zhangduo
Fix For: 1.0.0, 2.0.0
Attachments: 3149-trunk-v1.txt, HBASE-10201-0.98.patch,
HBASE-10201-0.98_1.patch, HBASE-10201-0.98_2.patch, HBASE-10201-0.99.patch,
HBASE-10201.patch, HBASE-10201_1.patch, HBASE-10201_10.patch,
HBASE-10201_11.patch, HBASE-10201_12.patch, HBASE-10201_13.patch,
HBASE-10201_13.patch, HBASE-10201_14.patch, HBASE-10201_15.patch,
HBASE-10201_16.patch, HBASE-10201_17.patch, HBASE-10201_18.patch,
HBASE-10201_19.patch, HBASE-10201_2.patch, HBASE-10201_3.patch,
HBASE-10201_4.patch, HBASE-10201_5.patch, HBASE-10201_6.patch,
HBASE-10201_7.patch, HBASE-10201_8.patch, HBASE-10201_9.patch,
compactions.png, count.png, io.png, memstore.png
Currently the flush decision is made using the aggregate size of all column
families. When large and small column families co-exist, this causes many
small flushes of the smaller CF. We need to make per-CF flush decisions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245248#comment-14245248
]
Varun Saxena commented on HBASE-12645:
--
Wonder why the result from Jenkins is not coming.
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12681) truncate_preserve comand fails with undefined method `getTable' error
Ashish Singhi created HBASE-12681:
-
Summary: truncate_preserve comand fails with undefined method
`getTable' error
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.99.2, 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12681) truncate_preserve comand fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HBASE-12681:
--
Status: Patch Available (was: Open)
truncate_preserve comand fails with undefined method `getTable' error
-
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.99.2, 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12681) truncate_preserve comand fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HBASE-12681:
--
Attachment: HBASE-12681.patch
truncate_preserve comand fails with undefined method `getTable' error
-
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245313#comment-14245313 ] Anoop Sam John commented on HBASE-12644: +1 Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 0.98.10 Attachments: HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12681) truncate_preserve comand fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245315#comment-14245315
]
Hadoop QA commented on HBASE-12681:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687033/HBASE-12681.patch
against master branch at commit a0e473730e2cd819e7442dbd2b332d7833755ba2.
ATTACHMENT ID: 12687033
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12066//console
This message is automatically generated.
truncate_preserve comand fails with undefined method `getTable' error
-
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12668) Adapt PayloadCarryingRpcController so it can also be used in async way
[ https://issues.apache.org/jira/browse/HBASE-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245322#comment-14245322 ] Jurriaan Mous commented on HBASE-12668: --- Thanks!! Is it possible to update 1.0.0-SNAPSHOT again? :) Adapt PayloadCarryingRpcController so it can also be used in async way -- Key: HBASE-12668 URL: https://issues.apache.org/jira/browse/HBASE-12668 Project: HBase Issue Type: Improvement Components: Client Reporter: Jurriaan Mous Assignee: Jurriaan Mous Fix For: 1.0.0, 2.0.0 Attachments: HBASE-12668-V1.patch, HBASE-12668-V1.patch, HBASE-12668.patch With the changes in HBASE-12597 it is possible to create a new RPC client. But in all places the BlockingRpcChannel is called with a PayloadCarryingRpcController. This controller is not usable in Async context because some methods are not supported at the moment. (See TimeLimitedRpcController for the methods that throw UnsupportedOperationException) This issue is about implementing these methods so PayloadCarryingRpcController can also be used in an async context and work the same in a sync context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245370#comment-14245370
]
Varun Saxena commented on HBASE-12645:
--
Ran all the test cases in local. Some are failing. Will investigate and if
required, upload a new patch
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12681) truncate_preserve command fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12681:
---
Summary: truncate_preserve command fails with undefined method `getTable'
error (was: truncate_preserve comand fails with undefined method `getTable'
error)
truncate_preserve command fails with undefined method `getTable' error
--
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12681) truncate_preserve command fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245418#comment-14245418
]
Ted Yu commented on HBASE-12681:
Thanks for the patch, Ashish.
truncate_preserve command fails with undefined method `getTable' error
--
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12681) truncate_preserve command fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12681:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
truncate_preserve command fails with undefined method `getTable' error
--
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245420#comment-14245420 ] Ted Yu commented on HBASE-12644: Integrated to master and branch-1 Thanks for the review, Anoop. Thanks for the contribution, Jerry. Can you prepare patch for 0.98 ? Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 0.98.10 Attachments: HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12644: --- Fix Version/s: 2.0.0 Hadoop Flags: Reviewed Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245433#comment-14245433 ] Ted Yu commented on HBASE-12644: Generating patch for 0.98 is not difficult. But, for existing 0.98 deployment, the persisted super user in labels table should be removed, right ? Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12644: --- Status: Open (was: Patch Available) Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.99.2, 0.98.8 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12644: --- Attachment: 12644-0.98.patch Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12681) truncate_preserve command fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245441#comment-14245441
]
Ashish Singhi commented on HBASE-12681:
---
Thanks for the review, Ted
truncate_preserve command fails with undefined method `getTable' error
--
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12659) Replace the method calls to grant and revoke in shell scripts with AccessControlClient
[ https://issues.apache.org/jira/browse/HBASE-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12659: --- Attachment: 12659-v2.patch Patch rebased on current master branch. Replace the method calls to grant and revoke in shell scripts with AccessControlClient --- Key: HBASE-12659 URL: https://issues.apache.org/jira/browse/HBASE-12659 Project: HBase Issue Type: Improvement Components: security, shell Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Attachments: 12659-v2.patch, HBASE-12659.patch Currently, the grant and revoke methods of ProtobufUtil methods are being used in ruby shell scripts. Using AccessControlClient methods instead will provide cleaner separation and easier maintenance. Thanks to [~mbertozzi] for making this suggestion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12659) Replace the method calls to grant and revoke in shell scripts with AccessControlClient
[
https://issues.apache.org/jira/browse/HBASE-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245458#comment-14245458
]
Hadoop QA commented on HBASE-12659:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687050/12659-v2.patch
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12687050
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12067//console
This message is automatically generated.
Replace the method calls to grant and revoke in shell scripts with
AccessControlClient
---
Key: HBASE-12659
URL: https://issues.apache.org/jira/browse/HBASE-12659
Project: HBase
Issue Type: Improvement
Components: security, shell
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Priority: Minor
Attachments: 12659-v2.patch, HBASE-12659.patch
Currently, the grant and revoke methods of ProtobufUtil methods are being
used in ruby shell scripts. Using AccessControlClient methods instead will
provide cleaner separation and easier maintenance. Thanks to [~mbertozzi] for
making this suggestion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12681) truncate_preserve command fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245457#comment-14245457
]
Hudson commented on HBASE-12681:
SUCCESS: Integrated in HBase-TRUNK #5916 (See
[https://builds.apache.org/job/HBase-TRUNK/5916/])
HBASE-12681 truncate_preserve command fails with undefined method 'getTable'
error (Ashish) (tedyu: rev 29c233e6e84cf5867610ca57659ca29df2792ee1)
* hbase-shell/src/main/ruby/hbase/admin.rb
truncate_preserve command fails with undefined method `getTable' error
--
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12659) Replace the method calls to grant and revoke in shell scripts with AccessControlClient
[ https://issues.apache.org/jira/browse/HBASE-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12659: --- Fix Version/s: 2.0.0 1.0.0 Hadoop Flags: Reviewed Integrated to branch-1 and master. Srikanth: Mind attaching patch for 0.98 ? Replace the method calls to grant and revoke in shell scripts with AccessControlClient --- Key: HBASE-12659 URL: https://issues.apache.org/jira/browse/HBASE-12659 Project: HBase Issue Type: Improvement Components: security, shell Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 1.0.0, 2.0.0 Attachments: 12659-v2.patch, HBASE-12659.patch Currently, the grant and revoke methods of ProtobufUtil methods are being used in ruby shell scripts. Using AccessControlClient methods instead will provide cleaner separation and easier maintenance. Thanks to [~mbertozzi] for making this suggestion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245483#comment-14245483 ] Hudson commented on HBASE-12644: SUCCESS: Integrated in HBase-1.0 #579 (See [https://builds.apache.org/job/HBase-1.0/579/]) HBASE-12644 Visibility Labels: issue with storing super users in labels table (Jerry He) (tedyu: rev 4d218bb2ede6c241c754ec0c9f15be218f8d11a6) * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ZKVisibilityLabelWatcher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestDefaultScanLabelGeneratorStack.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestEnforcingScanLabelGenerator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12681) truncate_preserve command fails with undefined method `getTable' error
[
https://issues.apache.org/jira/browse/HBASE-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245482#comment-14245482
]
Hudson commented on HBASE-12681:
SUCCESS: Integrated in HBase-1.0 #579 (See
[https://builds.apache.org/job/HBase-1.0/579/])
HBASE-12681 truncate_preserve command fails with undefined method 'getTable'
error (Ashish) (tedyu: rev d4853f0379fb649f4bbe914d60b1bc706b0d)
* hbase-shell/src/main/ruby/hbase/admin.rb
truncate_preserve command fails with undefined method `getTable' error
--
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0, 0.99.2
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12681.patch
{noformat}
hbase(main):002:0 truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245499#comment-14245499 ] Hudson commented on HBASE-12644: SUCCESS: Integrated in HBase-TRUNK #5917 (See [https://builds.apache.org/job/HBase-TRUNK/5917/]) HBASE-12644 Visibility Labels: issue with storing super users in labels table (Jerry He) (tedyu: rev b24518562adb5deb41a8780cead268cb163864d1) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestEnforcingScanLabelGenerator.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestDefaultScanLabelGeneratorStack.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ZKVisibilityLabelWatcher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultVisibilityLabelServiceImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/ExpAsStringVisibilityLabelServiceImpl.java Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12659) Replace the method calls to grant and revoke in shell scripts with AccessControlClient
[ https://issues.apache.org/jira/browse/HBASE-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245498#comment-14245498 ] Hudson commented on HBASE-12659: SUCCESS: Integrated in HBase-TRUNK #5917 (See [https://builds.apache.org/job/HBase-TRUNK/5917/]) HBASE-12659 Replace the method calls to grant and revoke in shell scripts with AccessControlClient (Srikanth Srungarapu) (tedyu: rev 65830b096b6f540b7e49ef590dac1ebe2491c126) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java * hbase-shell/src/main/ruby/hbase/security.rb * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlClient.java Replace the method calls to grant and revoke in shell scripts with AccessControlClient --- Key: HBASE-12659 URL: https://issues.apache.org/jira/browse/HBASE-12659 Project: HBase Issue Type: Improvement Components: security, shell Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 1.0.0, 2.0.0 Attachments: 12659-v2.patch, HBASE-12659.patch Currently, the grant and revoke methods of ProtobufUtil methods are being used in ruby shell scripts. Using AccessControlClient methods instead will provide cleaner separation and easier maintenance. Thanks to [~mbertozzi] for making this suggestion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12422) Use ConnectionFactory in HTable constructors
[
https://issues.apache.org/jira/browse/HBASE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12422:
--
Attachment: 12422v3.txt
Try this patch. Should fix the TestSecureLoadIncrementalHFilesSplitRecovery.
More work to do on the TestHBaseFsck failure (should have fixes one test
failure).
Here is fix for the TestSecureLoadIncremental...
45 -Table table = new HTable(conn.getConfiguration(),
getTableName());
46 -secureClient = new SecureBulkLoadClient(table);
47 -success = secureClient.bulkLoadHFiles(famPaths,
fsDelegationToken.getUserToken(),
48 - bulkToken, getLocation().getRegionInfo().getStartKey());
49 +try (Table table = conn.getTable(getTableName())) {
50 + secureClient = new SecureBulkLoadClient(table);
51 + success = secureClient.bulkLoadHFiles(famPaths,
fsDelegationToken.getUserToken(),
52 +bulkToken, getLocation().getRegionInfo().getStartKey());
53 +}
Use existing Connection rather than create new one w/ 'wrong' zk config -- so
fail to connect to zk.
Patch is big because some of the tests have been refactored to use new idiom
just so we do clean up of resources, action that was not there previous.
Also includes addition of timeout on tests so they fail rather than go zombie.
Lets see how this does one hadoopqa.
Use ConnectionFactory in HTable constructors
Key: HBASE-12422
URL: https://issues.apache.org/jira/browse/HBASE-12422
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Solomon Duskis
Assignee: stack
Priority: Minor
Fix For: 1.0.0, 2.0.0
Attachments: 12422v2.txt, 12422v3.txt, HBASE-12422.patch,
HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch
Use ConnectionFactory in HTable instead of ConnectionManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12668) Adapt PayloadCarryingRpcController so it can also be used in async way
[ https://issues.apache.org/jira/browse/HBASE-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245513#comment-14245513 ] stack commented on HBASE-12668: --- bq. Is it possible to update 1.0.0-SNAPSHOT again? Done. Adapt PayloadCarryingRpcController so it can also be used in async way -- Key: HBASE-12668 URL: https://issues.apache.org/jira/browse/HBASE-12668 Project: HBase Issue Type: Improvement Components: Client Reporter: Jurriaan Mous Assignee: Jurriaan Mous Fix For: 1.0.0, 2.0.0 Attachments: HBASE-12668-V1.patch, HBASE-12668-V1.patch, HBASE-12668.patch With the changes in HBASE-12597 it is possible to create a new RPC client. But in all places the BlockingRpcChannel is called with a PayloadCarryingRpcController. This controller is not usable in Async context because some methods are not supported at the moment. (See TimeLimitedRpcController for the methods that throw UnsupportedOperationException) This issue is about implementing these methods so PayloadCarryingRpcController can also be used in an async context and work the same in a sync context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12422) Use ConnectionFactory in HTable constructors
[
https://issues.apache.org/jira/browse/HBASE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245514#comment-14245514
]
Hadoop QA commented on HBASE-12422:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687057/12422v3.txt
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12687057
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12068//console
This message is automatically generated.
Use ConnectionFactory in HTable constructors
Key: HBASE-12422
URL: https://issues.apache.org/jira/browse/HBASE-12422
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Solomon Duskis
Assignee: stack
Priority: Minor
Fix For: 1.0.0, 2.0.0
Attachments: 12422v2.txt, 12422v3.txt, HBASE-12422.patch,
HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch
Use ConnectionFactory in HTable instead of ConnectionManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12422) Use ConnectionFactory in HTable constructors
[ https://issues.apache.org/jira/browse/HBASE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12422: -- Attachment: 12422v4.txt Rebase Use ConnectionFactory in HTable constructors Key: HBASE-12422 URL: https://issues.apache.org/jira/browse/HBASE-12422 Project: HBase Issue Type: Bug Components: Client Reporter: Solomon Duskis Assignee: stack Priority: Minor Fix For: 1.0.0, 2.0.0 Attachments: 12422v2.txt, 12422v3.txt, 12422v4.txt, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch Use ConnectionFactory in HTable instead of ConnectionManager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12659) Replace the method calls to grant and revoke in shell scripts with AccessControlClient
[ https://issues.apache.org/jira/browse/HBASE-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245517#comment-14245517 ] Hudson commented on HBASE-12659: SUCCESS: Integrated in HBase-1.0 #580 (See [https://builds.apache.org/job/HBase-1.0/580/]) HBASE-12659 Replace the method calls to grant and revoke in shell scripts with AccessControlClient (Srikanth Srungarapu) (tedyu: rev b7bb9d95051dcb691faf77ee85cc6641117bb594) * hbase-shell/src/main/ruby/hbase/security.rb * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlClient.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java Replace the method calls to grant and revoke in shell scripts with AccessControlClient --- Key: HBASE-12659 URL: https://issues.apache.org/jira/browse/HBASE-12659 Project: HBase Issue Type: Improvement Components: security, shell Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 1.0.0, 2.0.0 Attachments: 12659-v2.patch, HBASE-12659.patch Currently, the grant and revoke methods of ProtobufUtil methods are being used in ruby shell scripts. Using AccessControlClient methods instead will provide cleaner separation and easier maintenance. Thanks to [~mbertozzi] for making this suggestion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12668) Adapt PayloadCarryingRpcController so it can also be used in async way
[ https://issues.apache.org/jira/browse/HBASE-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245532#comment-14245532 ] Jurriaan Mous commented on HBASE-12668: --- I see you updated the master (2.0.0) but that will do for now since client api is the same. :) I can now in a quick test confirm that I can replace the RPC client with a Netty based one and start and stop a mini-cluster with it and it does all the normal sync communications correctly and it is able to do async communication through custom API calls. Later in the coming week I will try out some of the tests in HBase to see if the client works correctly with all current RpcClient tests and will then probably propose a new issue to integrate the AsyncRpcClient as an optional RpcClient in HBase itself. Adapt PayloadCarryingRpcController so it can also be used in async way -- Key: HBASE-12668 URL: https://issues.apache.org/jira/browse/HBASE-12668 Project: HBase Issue Type: Improvement Components: Client Reporter: Jurriaan Mous Assignee: Jurriaan Mous Fix For: 1.0.0, 2.0.0 Attachments: HBASE-12668-V1.patch, HBASE-12668-V1.patch, HBASE-12668.patch With the changes in HBASE-12597 it is possible to create a new RPC client. But in all places the BlockingRpcChannel is called with a PayloadCarryingRpcController. This controller is not usable in Async context because some methods are not supported at the moment. (See TimeLimitedRpcController for the methods that throw UnsupportedOperationException) This issue is about implementing these methods so PayloadCarryingRpcController can also be used in an async context and work the same in a sync context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12668) Adapt PayloadCarryingRpcController so it can also be used in async way
[ https://issues.apache.org/jira/browse/HBASE-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245540#comment-14245540 ] stack commented on HBASE-12668: --- bq. I see you updated the master (2.0.0) but that will do for now since client api is the same. Oh. Yeah. Wrong branch. Look again in 20mins. bq. Later in the coming week I will try out some of the tests in HBase to see if the client works correctly with all current RpcClient tests and will then probably propose a new issue to integrate the AsyncRpcClient as an optional RpcClient in HBase itself. Excellent. Would it be under, replace, or make use of our AsyncProcess in client package? In case you may not have noticed, our current client is a little 'heavyweight'; feel free pruning. Will there be a new Async Client Interface? Thanks. Adapt PayloadCarryingRpcController so it can also be used in async way -- Key: HBASE-12668 URL: https://issues.apache.org/jira/browse/HBASE-12668 Project: HBase Issue Type: Improvement Components: Client Reporter: Jurriaan Mous Assignee: Jurriaan Mous Fix For: 1.0.0, 2.0.0 Attachments: HBASE-12668-V1.patch, HBASE-12668-V1.patch, HBASE-12668.patch With the changes in HBASE-12597 it is possible to create a new RPC client. But in all places the BlockingRpcChannel is called with a PayloadCarryingRpcController. This controller is not usable in Async context because some methods are not supported at the moment. (See TimeLimitedRpcController for the methods that throw UnsupportedOperationException) This issue is about implementing these methods so PayloadCarryingRpcController can also be used in an async context and work the same in a sync context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12645:
--
Attachment: HBASE-12645.004.patch
Retry
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245543#comment-14245543
]
Varun Saxena commented on HBASE-12645:
--
[~stack], current patch will fail on some tests. Will upload a new patch after
running all the tests in my local setup.
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedErro
[
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245545#comment-14245545
]
stack commented on HBASE-12558:
---
Downed priority. No one working on it.
TestHCM.testClusterStatus Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
-
Key: HBASE-12558
URL: https://issues.apache.org/jira/browse/HBASE-12558
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Fix For: 1.0.0, 2.0.0
Attachments: 12558-master.patch, 12558.ignore.txt
Happens for me reliably on mac os x. I looked at fixing it. The listener is
not noticing the publish for whatever reason. Thats where I stopped.
{code}
java.lang.Exception: Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedError
[
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12558:
--
Priority: Major (was: Critical)
TestHCM.testClusterStatus Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
-
Key: HBASE-12558
URL: https://issues.apache.org/jira/browse/HBASE-12558
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Fix For: 1.0.0, 2.0.0
Attachments: 12558-master.patch, 12558.ignore.txt
Happens for me reliably on mac os x. I looked at fixing it. The listener is
not noticing the publish for whatever reason. Thats where I stopped.
{code}
java.lang.Exception: Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245547#comment-14245547 ] Jerry He commented on HBASE-12644: -- Hi, Ted If we want to do any cleaning of the super users from the labels table automatically, currently there is no way to tell the super users from the normal users who were explicitly granted 'system' label. Also for rolling upgrade to work continuously, we probably can not delete the super user entries in the labels table until the rolling upgrade is entirely completed. Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12679) Add HBaseInterfaceAudience.TOOLS and move some of the Public classes to LimitedPrivate
[ https://issues.apache.org/jira/browse/HBASE-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12679: -- Status: Patch Available (was: Open) Add HBaseInterfaceAudience.TOOLS and move some of the Public classes to LimitedPrivate -- Key: HBASE-12679 URL: https://issues.apache.org/jira/browse/HBASE-12679 Project: HBase Issue Type: Sub-task Components: Client Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 1.0.0, 2.0.0 Attachments: hbase-12679_v1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12679) Add HBaseInterfaceAudience.TOOLS and move some of the Public classes to LimitedPrivate
[ https://issues.apache.org/jira/browse/HBASE-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245551#comment-14245551 ] stack commented on HBASE-12679: --- +1 Looks great. Add HBaseInterfaceAudience.TOOLS and move some of the Public classes to LimitedPrivate -- Key: HBASE-12679 URL: https://issues.apache.org/jira/browse/HBASE-12679 Project: HBase Issue Type: Sub-task Components: Client Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 1.0.0, 2.0.0 Attachments: hbase-12679_v1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12675) Use interface methods in shell scripts
[ https://issues.apache.org/jira/browse/HBASE-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245553#comment-14245553 ] stack commented on HBASE-12675: --- You have a patch up your sleeve [~sduskis]? Use interface methods in shell scripts -- Key: HBASE-12675 URL: https://issues.apache.org/jira/browse/HBASE-12675 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 1.0.0, 2.0.0 There are places in the shell script code that use methods from HTable or HConnection or etc. This patch will fix at least some of them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12636) Avoid too many write operations on zookeeper in replication
[ https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12636: -- Status: Patch Available (was: Open) Will commit in next day unless objection. Avoid too many write operations on zookeeper in replication --- Key: HBASE-12636 URL: https://issues.apache.org/jira/browse/HBASE-12636 Project: HBase Issue Type: Improvement Affects Versions: 0.94.11 Reporter: Liu Shaohui Assignee: Liu Shaohui Labels: replication Fix For: 1.0.0 Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff In our production cluster, we found there are about over 1k write operations per second on zookeeper from hbase replication. The reason is that the replication source will write the log position to zookeeper for every edit shipping. If the current replicating WAL is just the WAL that regionserver is writing to, each skipping will be very small but the frequency is very high, which causes many write operations on zookeeper. A simple solution is that writing log position to zookeeper when position diff or skipped edit number is larger than a threshold, not every edit shipping. Suggestions are welcomed, thx~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12567) Track remaining issues for HBase-1.0
[ https://issues.apache.org/jira/browse/HBASE-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245559#comment-14245559 ] stack commented on HBASE-12567: --- Made this a blocker but this could be resolved? The remaining issue is itself the only blocker. Track remaining issues for HBase-1.0 Key: HBASE-12567 URL: https://issues.apache.org/jira/browse/HBASE-12567 Project: HBase Issue Type: Task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 1.0.0 Just for tracking (a couple of) remaining jiras for 1.0. This will track only absolute criticals. I am thinking that 0.99.2 will be the last 0.99 release. We should do 1.0.0RC shortly afterwards. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12567) Track remaining issues for HBase-1.0
[ https://issues.apache.org/jira/browse/HBASE-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12567: -- Priority: Blocker (was: Major) Track remaining issues for HBase-1.0 Key: HBASE-12567 URL: https://issues.apache.org/jira/browse/HBASE-12567 Project: HBase Issue Type: Task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 1.0.0 Just for tracking (a couple of) remaining jiras for 1.0. This will track only absolute criticals. I am thinking that 0.99.2 will be the last 0.99 release. We should do 1.0.0RC shortly afterwards. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12679) Add HBaseInterfaceAudience.TOOLS and move some of the Public classes to LimitedPrivate
[
https://issues.apache.org/jira/browse/HBASE-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245561#comment-14245561
]
Hadoop QA commented on HBASE-12679:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12686773/hbase-12679_v1.patch
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12686773
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12071//console
This message is automatically generated.
Add HBaseInterfaceAudience.TOOLS and move some of the Public classes to
LimitedPrivate
--
Key: HBASE-12679
URL: https://issues.apache.org/jira/browse/HBASE-12679
Project: HBase
Issue Type: Sub-task
Components: Client
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 1.0.0, 2.0.0
Attachments: hbase-12679_v1.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12563) After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility
[ https://issues.apache.org/jira/browse/HBASE-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245562#comment-14245562 ] stack commented on HBASE-12563: --- [~talat] Their info port is in zk IIRC. If for unit tests, you could iterate the running daemons -- you can get the set from the minihbasecluster instance -- and this way ask each daemon what their info port is? After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility - Key: HBASE-12563 URL: https://issues.apache.org/jira/browse/HBASE-12563 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0, 0.99.1 Reporter: Talat UYARER Assignee: Talat UYARER Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: HBASE-12563-v2.patch, HBASE-12563-v3.patch, HBASE-12563.patch When I use startMiniHBaseCluster method. It starts up a Mini Cluster for the tests. While starting It changes some configuration. For example Master's port or Region Server's. After Cluster starting We should set its new configuration to HbaseTestingUtils conf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedError
[
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12558:
--
Fix Version/s: (was: 1.0.0)
TestHCM.testClusterStatus Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
-
Key: HBASE-12558
URL: https://issues.apache.org/jira/browse/HBASE-12558
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Fix For: 2.0.0
Attachments: 12558-master.patch, 12558.ignore.txt
Happens for me reliably on mac os x. I looked at fixing it. The listener is
not noticing the publish for whatever reason. Thats where I stopped.
{code}
java.lang.Exception: Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12422) Use ConnectionFactory in HTable constructors
[
https://issues.apache.org/jira/browse/HBASE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245569#comment-14245569
]
Hadoop QA commented on HBASE-12422:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687060/12422v4.txt
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12687060
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12069//console
This message is automatically generated.
Use ConnectionFactory in HTable constructors
Key: HBASE-12422
URL: https://issues.apache.org/jira/browse/HBASE-12422
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Solomon Duskis
Assignee: stack
Priority: Minor
Fix For: 1.0.0, 2.0.0
Attachments: 12422v2.txt, 12422v3.txt, 12422v4.txt,
HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch,
HBASE-12422.patch
Use ConnectionFactory in HTable instead of ConnectionManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12446) [list | abort] Compactions
[ https://issues.apache.org/jira/browse/HBASE-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245572#comment-14245572 ] stack commented on HBASE-12446: --- [~esteban] FYI [list | abort] Compactions -- Key: HBASE-12446 URL: https://issues.apache.org/jira/browse/HBASE-12446 Project: HBase Issue Type: New Feature Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Fix For: 1.0.0 Original Estimate: 168h Remaining Estimate: 168h In some cases, we would need to quickly reduce load on a server without killing it. Compactions is one of the critical processes which takes up a lot of CPU and disk IOPS. We should have a way to list compactions given the regionserver and abort compactions given regionserver and compaction id. And additionally abort all compactions. Pardon me if there was already a similar Jira, I'd be happy to merge this there. The current code handles interrupts. We should be able to interrupt the thread that is performing the compaction and abort it from either the UI or from the command line. This Jira is targeted to expose an admin function to perform such a task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245579#comment-14245579 ] Jerry He commented on HBASE-12644: -- This fix prevents the described issue from happening in the future. But if there is a previous deployment with super users in the labels table already, it looks like we can not clean them because of the above reasons. It will not break any upgrade. What about a release note that document this? Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12446) [list | abort] Compactions
[ https://issues.apache.org/jira/browse/HBASE-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245585#comment-14245585 ] Esteban Gutierrez commented on HBASE-12446: --- Yeah, I'm working on HBASE-6028, my prototype handles only the case to cancel all compactions in progress for now, but I think we can move it as sub task to handle the shell commands that expose this functionality. Does it sounds good for you [~manukranthk]? [list | abort] Compactions -- Key: HBASE-12446 URL: https://issues.apache.org/jira/browse/HBASE-12446 Project: HBase Issue Type: New Feature Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Fix For: 1.0.0 Original Estimate: 168h Remaining Estimate: 168h In some cases, we would need to quickly reduce load on a server without killing it. Compactions is one of the critical processes which takes up a lot of CPU and disk IOPS. We should have a way to list compactions given the regionserver and abort compactions given regionserver and compaction id. And additionally abort all compactions. Pardon me if there was already a similar Jira, I'd be happy to merge this there. The current code handles interrupts. We should be able to interrupt the thread that is performing the compaction and abort it from either the UI or from the command line. This Jira is targeted to expose an admin function to perform such a task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12668) Adapt PayloadCarryingRpcController so it can also be used in async way
[ https://issues.apache.org/jira/browse/HBASE-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245616#comment-14245616 ] Jurriaan Mous commented on HBASE-12668: --- I am trying to do this step by step to be sure each step is correct. So I would first propose an async RpcClient which works with all current sync API. Nothing fancy, just a main AsyncRpcClient class, A SaslHandler, a new Call class (Or adapt current one), and a Response Handler and make sure it can be loaded with config to replace main RpcClient. And if it is found to be working great and better performant it could become the default and the current one could be removed. In an async RpcClient a lot of stuff in AsyncProcess would not be needed. (Netty handles that internally while handling the communication) AsyncProcess now seems to be a complex piece of code to simulate async calls by throwing sync calls onto new threads and collects all responses. But it is needed while the old blocking RpcClient exists. Does it sound like a good path to do this step by step? And if AsyncRpcClient works out as an optional RpcClient, that we evaluate the next step of removing the sync RpcClient. When removed it is possible to move over all the AsyncProcess code and introduce a new async RpcClient interface. And then it would become possible to introduce a new optional Async api for Table etc. Adapt PayloadCarryingRpcController so it can also be used in async way -- Key: HBASE-12668 URL: https://issues.apache.org/jira/browse/HBASE-12668 Project: HBase Issue Type: Improvement Components: Client Reporter: Jurriaan Mous Assignee: Jurriaan Mous Fix For: 1.0.0, 2.0.0 Attachments: HBASE-12668-V1.patch, HBASE-12668-V1.patch, HBASE-12668.patch With the changes in HBASE-12597 it is possible to create a new RPC client. But in all places the BlockingRpcChannel is called with a PayloadCarryingRpcController. This controller is not usable in Async context because some methods are not supported at the moment. (See TimeLimitedRpcController for the methods that throw UnsupportedOperationException) This issue is about implementing these methods so PayloadCarryingRpcController can also be used in an async context and work the same in a sync context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245670#comment-14245670
]
Hadoop QA commented on HBASE-12645:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687067/HBASE-12645.004.patch
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12687067
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation
org.apache.hadoop.hbase.snapshot.TestExportSnapshot
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot
org.apache.hadoop.hbase.util.TestMergeTable
org.apache.hadoop.hbase.master.handler.TestTableDeleteFamilyHandler
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas
org.apache.hadoop.hbase.TestMultiVersions
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12070//console
This message is automatically generated.
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
[jira] [Commented] (HBASE-12636) Avoid too many write operations on zookeeper in replication
[
https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245673#comment-14245673
]
Hadoop QA commented on HBASE-12636:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685233/HBASE-12635-v2.diff
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12685233
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12072//console
This message is automatically generated.
Avoid too many write operations on zookeeper in replication
---
Key: HBASE-12636
URL: https://issues.apache.org/jira/browse/HBASE-12636
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.11
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Labels: replication
Fix For: 1.0.0
Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff
In our production cluster, we found there are about over 1k write operations
per second on zookeeper from hbase replication. The reason is that the
replication source will write the log position to zookeeper for every edit
shipping. If the current replicating WAL is just the WAL that regionserver is
writing to, each skipping will be very small but the frequency is very high,
which causes many write operations on zookeeper.
A simple solution is that writing log position to zookeeper when position
diff or skipped edit number is larger than a threshold, not every edit
shipping.
Suggestions are welcomed, thx~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245767#comment-14245767 ] Jerry He commented on HBASE-12644: -- Hi, [~apurtell] Do you have any comment or suggestion? Thanks. Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12659) Replace the method calls to grant and revoke in shell scripts with AccessControlClient
[ https://issues.apache.org/jira/browse/HBASE-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-12659: Attachment: HBASE-12659_0.98.patch Attaching the patch for 0.98 branch. Thanks Ted for pushing the fix. Replace the method calls to grant and revoke in shell scripts with AccessControlClient --- Key: HBASE-12659 URL: https://issues.apache.org/jira/browse/HBASE-12659 Project: HBase Issue Type: Improvement Components: security, shell Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 1.0.0, 2.0.0 Attachments: 12659-v2.patch, HBASE-12659.patch, HBASE-12659_0.98.patch Currently, the grant and revoke methods of ProtobufUtil methods are being used in ruby shell scripts. Using AccessControlClient methods instead will provide cleaner separation and easier maintenance. Thanks to [~mbertozzi] for making this suggestion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12659) Replace the method calls to grant and revoke in shell scripts with AccessControlClient
[
https://issues.apache.org/jira/browse/HBASE-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245793#comment-14245793
]
Hadoop QA commented on HBASE-12659:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687091/HBASE-12659_0.98.patch
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12687091
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12073//console
This message is automatically generated.
Replace the method calls to grant and revoke in shell scripts with
AccessControlClient
---
Key: HBASE-12659
URL: https://issues.apache.org/jira/browse/HBASE-12659
Project: HBase
Issue Type: Improvement
Components: security, shell
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Priority: Minor
Fix For: 1.0.0, 2.0.0
Attachments: 12659-v2.patch, HBASE-12659.patch, HBASE-12659_0.98.patch
Currently, the grant and revoke methods of ProtobufUtil methods are being
used in ruby shell scripts. Using AccessControlClient methods instead will
provide cleaner separation and easier maintenance. Thanks to [~mbertozzi] for
making this suggestion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Saxena updated HBASE-12645:
-
Attachment: HBASE-12645.005.patch
Uploading a new patch. With this patch, all the test cases passed in my local
setup. Let us see how Jenkins responds.
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.005.patch,
HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Saxena updated HBASE-12645:
-
Status: Open (was: Patch Available)
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.005.patch,
HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Saxena updated HBASE-12645:
-
Status: Patch Available (was: Open)
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.005.patch,
HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-2502) HBase won't bind to designated interface when more than one network interface is available
[ https://issues.apache.org/jira/browse/HBASE-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245806#comment-14245806 ] Carter Shanklin commented on HBASE-2502: One of our users reports this is fixed by HBASE-8148. Can this issue be closed? HBase won't bind to designated interface when more than one network interface is available -- Key: HBASE-2502 URL: https://issues.apache.org/jira/browse/HBASE-2502 Project: HBase Issue Type: Bug Reporter: stack See this message by Michael Segel up on the list: http://www.mail-archive.com/hbase-user@hadoop.apache.org/msg10042.html This comes up from time to time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12636) Avoid too many write operations on zookeeper in replication
[ https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245817#comment-14245817 ] Lars Hofhansl commented on HBASE-12636: --- Is it really OK to double replicate data? It's not doing that now, right? It's especially bad when two clusters are setup in master-master replication and both are active. If you all feel that's OK, let's commit... -0 from me. Avoid too many write operations on zookeeper in replication --- Key: HBASE-12636 URL: https://issues.apache.org/jira/browse/HBASE-12636 Project: HBase Issue Type: Improvement Affects Versions: 0.94.11 Reporter: Liu Shaohui Assignee: Liu Shaohui Labels: replication Fix For: 1.0.0 Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff In our production cluster, we found there are about over 1k write operations per second on zookeeper from hbase replication. The reason is that the replication source will write the log position to zookeeper for every edit shipping. If the current replicating WAL is just the WAL that regionserver is writing to, each skipping will be very small but the frequency is very high, which causes many write operations on zookeeper. A simple solution is that writing log position to zookeeper when position diff or skipped edit number is larger than a threshold, not every edit shipping. Suggestions are welcomed, thx~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12636) Avoid too many write operations on zookeeper in replication
[ https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245821#comment-14245821 ] stack commented on HBASE-12636: --- bq. Is it really OK to double replicate data? It's not doing that now, right? The duplicate will just overwrite. Counters will be slightly off. I was thinking this only repercussion. Better than buckling your local zk ensemble with updates I'd say. Fine holding on commit till gets a bit more attention. Avoid too many write operations on zookeeper in replication --- Key: HBASE-12636 URL: https://issues.apache.org/jira/browse/HBASE-12636 Project: HBase Issue Type: Improvement Affects Versions: 0.94.11 Reporter: Liu Shaohui Assignee: Liu Shaohui Labels: replication Fix For: 1.0.0 Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff In our production cluster, we found there are about over 1k write operations per second on zookeeper from hbase replication. The reason is that the replication source will write the log position to zookeeper for every edit shipping. If the current replicating WAL is just the WAL that regionserver is writing to, each skipping will be very small but the frequency is very high, which causes many write operations on zookeeper. A simple solution is that writing log position to zookeeper when position diff or skipped edit number is larger than a threshold, not every edit shipping. Suggestions are welcomed, thx~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12636) Avoid too many write operations on zookeeper in replication
[ https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245824#comment-14245824 ] Lars Hofhansl commented on HBASE-12636: --- Was thinking that if both cluster are active a write that is repeated much later will might cause issues. We can also drive down the frequency of the checks... Maybe? I.e. if the frequency is too high, just check a little less often and let more data accumulate. Avoid too many write operations on zookeeper in replication --- Key: HBASE-12636 URL: https://issues.apache.org/jira/browse/HBASE-12636 Project: HBase Issue Type: Improvement Affects Versions: 0.94.11 Reporter: Liu Shaohui Assignee: Liu Shaohui Labels: replication Fix For: 1.0.0 Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff In our production cluster, we found there are about over 1k write operations per second on zookeeper from hbase replication. The reason is that the replication source will write the log position to zookeeper for every edit shipping. If the current replicating WAL is just the WAL that regionserver is writing to, each skipping will be very small but the frequency is very high, which causes many write operations on zookeeper. A simple solution is that writing log position to zookeeper when position diff or skipped edit number is larger than a threshold, not every edit shipping. Suggestions are welcomed, thx~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12422) Use ConnectionFactory in HTable constructors
[ https://issues.apache.org/jira/browse/HBASE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12422: -- Attachment: 12422v5.txt Looks like the fixup in the test deletetable in TestHBaseFsck fixed all failures, not just a few. The other failure looks like my changing a signature in a test override of incremental bulk load. Lets see how this patch does. Use ConnectionFactory in HTable constructors Key: HBASE-12422 URL: https://issues.apache.org/jira/browse/HBASE-12422 Project: HBase Issue Type: Bug Components: Client Reporter: Solomon Duskis Assignee: stack Priority: Minor Fix For: 1.0.0, 2.0.0 Attachments: 12422v2.txt, 12422v3.txt, 12422v4.txt, 12422v5.txt, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch Use ConnectionFactory in HTable instead of ConnectionManager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245838#comment-14245838
]
Hadoop QA commented on HBASE-12645:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687092/HBASE-12645.005.patch
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12687092
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12074//console
This message is automatically generated.
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.005.patch,
HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
[jira] [Commented] (HBASE-12680) Refactor base ClusterManager in HBase to not have the notion of sending a signal.
[ https://issues.apache.org/jira/browse/HBASE-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245840#comment-14245840 ] Enis Soztutar commented on HBASE-12680: --- This could have still been achieved with just overriding the methods in the custom ClusterManager. But the patch won't hurt. It removes signal, which is not used from ChaosMonkey anyway. Refactor base ClusterManager in HBase to not have the notion of sending a signal. - Key: HBASE-12680 URL: https://issues.apache.org/jira/browse/HBASE-12680 Project: HBase Issue Type: Improvement Components: integration tests Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Yi Deng Assignee: Yi Deng Priority: Minor Labels: integration-test Fix For: 1.0.0, 2.0.0, 0.98.7 Attachments: 2.0-0001-Move-signal-related-code-from-ClusterManager.patch, 2.0-0002-Move-signal-related-code-from-ClusterManager.patch, 2.0-0003-Move-signal-related-code-from-ClusterManager.patch In some situation, we need to run the integration test under some system not using ssh commands. Move the ssh/signal related parts out of ClusterManager and make them as part of the implementation details of HBaseClusterManager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12645) HBaseTestingUtility is using ${$HOME} for rootDir
[
https://issues.apache.org/jira/browse/HBASE-12645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245841#comment-14245841
]
Varun Saxena commented on HBASE-12645:
--
[~stack], [~ndimiduk], [~enis], kindly review.
HBaseTestingUtility is using ${$HOME} for rootDir
-
Key: HBASE-12645
URL: https://issues.apache.org/jira/browse/HBASE-12645
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 1.0.0
Reporter: Nick Dimiduk
Assignee: Varun Saxena
Priority: Critical
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12645.002.patch, HBASE-12645.003.patch,
HBASE-12645.004.patch, HBASE-12645.004.patch, HBASE-12645.005.patch,
HBASE-12645.patch
I noticed this while running tests on branch-1
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 sec
FAILURE! - in
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits Time
elapsed: 0.009 sec ERROR!
java.io.FileNotFoundException: Destination exists and is not a directory:
/homes/hortonnd/hbase
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:423)
at
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:588)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1053)
at
org.apache.hadoop.hbase.regionserver.wal.TestReadOldRootAndMetaEdits.setupBeforeClass(TestReadOldRootAndMetaEdits.java:70)
{noformat}
Either the testing utility has a regression or there's a config regression in
this test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12682) compaction thread throttle value is not correct in hbase-default.xml
Jerry He created HBASE-12682:
Summary: compaction thread throttle value is not correct in
hbase-default.xml
Key: HBASE-12682
URL: https://issues.apache.org/jira/browse/HBASE-12682
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0
Reporter: Jerry He
Assignee: Jerry He
Fix For: 2.0.0
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in the
documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one for
large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value is
appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I miss anyting?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12682) compaction thread throttle value is not correct in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerry He updated HBASE-12682:
-
Description:
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in the
documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one for
large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value is
appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I missing anything?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
was:
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in the
documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one for
large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value is
appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I missing anyting?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
compaction thread throttle value is not correct in hbase-default.xml
Key: HBASE-12682
URL: https://issues.apache.org/jira/browse/HBASE-12682
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0
Reporter: Jerry He
Assignee: Jerry He
Fix For: 2.0.0
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in
the documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one
for large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value
is appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I missing anything?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12682) compaction thread throttle value is not correct in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerry He updated HBASE-12682:
-
Description:
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in the
documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one for
large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value is
appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I missing anyting?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
was:
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in the
documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one for
large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value is
appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I miss anyting?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
compaction thread throttle value is not correct in hbase-default.xml
Key: HBASE-12682
URL: https://issues.apache.org/jira/browse/HBASE-12682
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0
Reporter: Jerry He
Assignee: Jerry He
Fix For: 2.0.0
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in
the documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one
for large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value
is appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I missing anyting?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12682) compaction thread throttle value is not correct in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerry He updated HBASE-12682:
-
Attachment: HBASE-12682-master.patch
compaction thread throttle value is not correct in hbase-default.xml
Key: HBASE-12682
URL: https://issues.apache.org/jira/browse/HBASE-12682
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0
Reporter: Jerry He
Assignee: Jerry He
Fix For: 2.0.0
Attachments: HBASE-12682-master.patch
While doing some compaction tuning and looking up some of the parameters, I
noticed that hbase.regionserver.thread.compaction.throttle default value in
the documentation and in hbase-site.xml seems to be off the mark.
{code}
property
namehbase.regionserver.thread.compaction.throttle/name
value2560/value
descriptionThere are two different thread pools for compactions, one
for large compactions and
the other for small compactions. This helps to keep compaction of lean
tables (such as
systemitemhbase:meta/systemitem) fast. If a compaction is larger
than this threshold, it
goes into the large compaction pool. In most cases, the default value
is appropriate. Default:
2 x hbase.hstore.compaction.max x hbase.hregion.memstore.flush.size
(which defaults to 128).
The value field assumes that the value of
hbase.hregion.memstore.flush.size is unchanged from
the default./description
/property
{code}
It should be in bytes. Or am I missing anything?
It is not a problem in 0.98 since the property is not in hbase-site.xml over
there. It is obtained dynamically:
{code}
throttlePoint = conf.getLong(hbase.regionserver.thread.compaction.throttle,
2 * maxFilesToCompact * storeConfigInfo.getMemstoreFlushSize());
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12683) Compilation with hadoop-2.7.0-SNAPSHOT is broken
Enis Soztutar created HBASE-12683: - Summary: Compilation with hadoop-2.7.0-SNAPSHOT is broken Key: HBASE-12683 URL: https://issues.apache.org/jira/browse/HBASE-12683 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 1.0.0, 2.0.0, 0.98.10 HADOOP-11389 changed ReflectionUtils.printThreadInfo() signature, which breaks compilation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12644) Visibility Labels: issue with storing super users in labels table
[ https://issues.apache.org/jira/browse/HBASE-12644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245862#comment-14245862 ] Anoop Sam John commented on HBASE-12644: Will there be some issue if we keep the existing super user auth info as is in labels table? I think no issue. With this patch, we will ensure on start of the system we wont add the super users from now on (new deployments). But the flow which checks for SYSTEM label presence for the user, we have added the super users check new way. This should help. Issue is when a super user is removed later, from the conf. At that time another super user can remove the user auth... This needs a manual step. We can document that clearly. Other than that all is well. Right? Visibility Labels: issue with storing super users in labels table - Key: HBASE-12644 URL: https://issues.apache.org/jira/browse/HBASE-12644 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8, 0.99.2 Reporter: Jerry He Assignee: Jerry He Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: 12644-0.98.patch, HBASE-12644-master-v2.patch, HBASE-12644-master-v3.patch, HBASE-12644-master.patch Super users have all the permissions for ACL and Visibility labels. They are defined in hbase-site.xml. Currently in VisibilityController, we persist super user with their system permission in hbase:labels. This makes change in super user difficult. There are two issues: In the current DefaultVisibilityLabelServiceImpl.addSystemLabel, we only add super user when we initially create the 'system' label. No additional update after that even if super user changed. See code for details. Additionally, there is no mechanism to remove any super user from the labels table. We probably should not persist super users in the labels table. They are in hbase-site.xml and can just stay in labelsCache and used from labelsCache after retrieval by Visibility Controller. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12422) Use ConnectionFactory in HTable constructors
[
https://issues.apache.org/jira/browse/HBASE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245867#comment-14245867
]
Hadoop QA commented on HBASE-12422:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12687100/12422v5.txt
against master branch at commit 65830b096b6f540b7e49ef590dac1ebe2491c126.
ATTACHMENT ID: 12687100
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12075//console
This message is automatically generated.
Use ConnectionFactory in HTable constructors
Key: HBASE-12422
URL: https://issues.apache.org/jira/browse/HBASE-12422
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Solomon Duskis
Assignee: stack
Priority: Minor
Fix For: 1.0.0, 2.0.0
Attachments: 12422v2.txt, 12422v3.txt, 12422v4.txt, 12422v5.txt,
HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch, HBASE-12422.patch,
HBASE-12422.patch
Use ConnectionFactory in HTable instead of ConnectionManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12683) Compilation with hadoop-2.7.0-SNAPSHOT is broken
[ https://issues.apache.org/jira/browse/HBASE-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12683: -- Attachment: hbase-12683_v1.patch Added reflection for the method call. Compilation with hadoop-2.7.0-SNAPSHOT is broken Key: HBASE-12683 URL: https://issues.apache.org/jira/browse/HBASE-12683 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: hbase-12683_v1.patch HADOOP-11389 changed ReflectionUtils.printThreadInfo() signature, which breaks compilation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12683) Compilation with hadoop-2.7.0-SNAPSHOT is broken
[ https://issues.apache.org/jira/browse/HBASE-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12683: -- Status: Patch Available (was: Open) Compilation with hadoop-2.7.0-SNAPSHOT is broken Key: HBASE-12683 URL: https://issues.apache.org/jira/browse/HBASE-12683 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: hbase-12683_v1.patch HADOOP-11389 changed ReflectionUtils.printThreadInfo() signature, which breaks compilation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
