[jira] [Commented] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093812#comment-14093812
]
Hadoop QA commented on HBASE-11685:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661133/HBASE-11685-trunk-v5.diff
against trunk revision .
ATTACHMENT ID: 12661133
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestMetaTableAccessorNoCluster
org.apache.hadoop.hbase.security.token.TestZKSecretWatcher
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10395//console
This message is automatically generated.
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff, HBASE-11685-trunk-v2.diff,
> HBASE-11685-trunk-v3.diff, HBASE-11685-trunk-v4.diff,
> HBASE-11685-trunk-v5.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11660) Make WAL reader follow contract for java.io.InputStream.available()
[
https://issues.apache.org/jira/browse/HBASE-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093778#comment-14093778
]
Hadoop QA commented on HBASE-11660:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659674/hbase-11660.01.patch
against trunk revision .
ATTACHMENT ID: 12659674
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.codec.TestCellCodec
org.apache.hadoop.hbase.codec.TestKeyValueCodecWithTags
org.apache.hadoop.hbase.codec.TestKeyValueCodec
org.apache.hadoop.hbase.codec.TestCellCodecWithTags
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10396//console
This message is automatically generated.
> Make WAL reader follow contract for java.io.InputStream.available()
> ---
>
> Key: HBASE-11660
> URL: https://issues.apache.org/jira/browse/HBASE-11660
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.99.0, 2.0.0, 0.98.6
>Reporter: Eric Hanson
>Priority: Minor
> Attachments: hbase-11660.01.patch
>
>
> In the process of building support to running HBase on Microsoft Azure
> HDInsight, I hit an issue in the HBase WAL reading process that took a lot of
> time to debug. The WAL reading code depends on available() for the log
> InputStream never returing 0 until end of file. This is not the same as the
> contract in java.io.InputStream for available.
> To prevent future grief for others that may want to port HBase onto storage
> systems other than HDFS, I propose to change the HBase WAL reader so it does
> not assume that EOF has been reached when available() == 0. It instead would
> treat available only as described in InputStream, i.e. available() is merely
> the number of bytes that could be read from the stream without blocking. That
> could be 0 even before EOF.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9531) a command line (hbase shell) interface to retreive the replication metrics and show replication lag
[
https://issues.apache.org/jira/browse/HBASE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093772#comment-14093772
]
Hadoop QA commented on HBASE-9531:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661123/HBASE-9531-master-v3.patch
against trunk revision .
ATTACHMENT ID: 12661123
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ new java.lang.String[] { "PeerID", "AgeOfLastShippedOp",
"SizeOfLogQueue", "TimeStampOfLastShippedOp", "ReplicationLag", });
+ new java.lang.String[] { "NumberOfRequests",
"TotalNumberOfRequests", "UsedHeapMB", "MaxHeapMB", "RegionLoads",
"Coprocessors", "ReportStartTime", "ReportEndTime", "InfoServerPort",
"ReplLoadSource", "ReplLoadSink", });
+if
(!@admin.getConfiguration().getBoolean(org.apache.hadoop.hbase.HConstants::REPLICATION_ENABLE_KEY,
org.apache.hadoop.hbase.HConstants::REPLICATION_ENABLE_DEFAULT))
+rSinkString << ", TimeStampsOfLastAppliedOp=" +
(java.util.Date.new(rLoadSink.getTimeStampsOfLastAppliedOp())).toString()
+ rSourceString << ", TimeStampsOfLastShippedOp=" +
(java.util.Date.new(rLoadSource.getTimeStampOfLastShippedOp())).toString()
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10393//console
This message is automatically generated.
> a command line (hbase shell) interface to retreive the replication metrics
> and show replication lag
> ---
>
> Key: HBASE-9531
> URL: https://issues.apache.org/jira/browse/HBASE-9531
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Affects Versions: 0.99.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 0.99.0, 0.98.6
>
> Attachments: HBASE-9531-master-v1.patch, HBASE-9531-master-v1.patch,
> HBASE-9531-master-v1.patch, HBASE-9531-master-v2.patch,
> HBASE-9531-master-v3.patch, HBASE-9531-trunk-v0.patch,
> HBASE-9531-trunk-v0.patch
>
>
> This jira is to provide a command line (hbase shell) interface to retreive
> the replication metrics info such as:ageOfLastShippedOp,
> timeStampsOfLastShippedOp, sizeOfLogQueue ageOfLastAppliedOp, and
> timeStampsOfLastAppliedOp. And also to provide a point of time info of the
> lag of replication(source only)
> Understand that hbase is using Hadoop
> metrics(http://hbase.a
[jira] [Updated] (HBASE-11722) Document new shortcut commands introduced by HBASE-11649
[ https://issues.apache.org/jira/browse/HBASE-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11722: Status: Open (was: Patch Available) > Document new shortcut commands introduced by HBASE-11649 > > > Key: HBASE-11722 > URL: https://issues.apache.org/jira/browse/HBASE-11722 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 0.98.4 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 2.0.0, 0.98.5 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11722) Document new shortcut commands introduced by HBASE-11649
[ https://issues.apache.org/jira/browse/HBASE-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-11722. - Resolution: Duplicate Not sure how that happened... > Document new shortcut commands introduced by HBASE-11649 > > > Key: HBASE-11722 > URL: https://issues.apache.org/jira/browse/HBASE-11722 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 0.98.4 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 2.0.0, 0.98.5 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11722) Document new shortcut commands introduced by HBASE-11649
[ https://issues.apache.org/jira/browse/HBASE-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11722: Status: Patch Available (was: Open) > Document new shortcut commands introduced by HBASE-11649 > > > Key: HBASE-11722 > URL: https://issues.apache.org/jira/browse/HBASE-11722 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 0.98.4 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 2.0.0, 0.98.5 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11723) Document all options of bin/hbase command
[ https://issues.apache.org/jira/browse/HBASE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11723: Attachment: HBASE-11723.patch Documented general syntax as well as "clean", "pe", "ltt", which are new. > Document all options of bin/hbase command > - > > Key: HBASE-11723 > URL: https://issues.apache.org/jira/browse/HBASE-11723 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11723.patch > > > The bin/hbase command is not documented fully in the Ref Guide: > http://hbase.apache.org/book.html#tools > Specifically a few new options were added in HBASE-11649 and need to be > documented. Also the generic usage instructions need to be there. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11723) Document all options of bin/hbase command
[ https://issues.apache.org/jira/browse/HBASE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11723: Status: Patch Available (was: Open) > Document all options of bin/hbase command > - > > Key: HBASE-11723 > URL: https://issues.apache.org/jira/browse/HBASE-11723 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11723.patch > > > The bin/hbase command is not documented fully in the Ref Guide: > http://hbase.apache.org/book.html#tools > Specifically a few new options were added in HBASE-11649 and need to be > documented. Also the generic usage instructions need to be there. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11723) Document all options of bin/hbase command
Misty Stanley-Jones created HBASE-11723: --- Summary: Document all options of bin/hbase command Key: HBASE-11723 URL: https://issues.apache.org/jira/browse/HBASE-11723 Project: HBase Issue Type: Task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones The bin/hbase command is not documented fully in the Ref Guide: http://hbase.apache.org/book.html#tools Specifically a few new options were added in HBASE-11649 and need to be documented. Also the generic usage instructions need to be there. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11708) RegionSplitter incorrectly calculates splitcount
[
https://issues.apache.org/jira/browse/HBASE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093743#comment-14093743
]
Hadoop QA commented on HBASE-11708:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661124/HBASE_11708-v3.patch
against trunk revision .
ATTACHMENT ID: 12661124
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10391//console
This message is automatically generated.
> RegionSplitter incorrectly calculates splitcount
>
>
> Key: HBASE-11708
> URL: https://issues.apache.org/jira/browse/HBASE-11708
> Project: HBase
> Issue Type: Bug
> Components: Admin, util
>Affects Versions: 0.96.2, 0.98.1
>Reporter: Sean Busbey
>Assignee: louis hust
>Priority: Minor
> Labels: beginner
> Attachments: HBASE_11708-v2.patch, HBASE_11708-v3.patch,
> HBASE_11708.patch
>
>
> From discussion on HBASE-11627:
> {quote}
> And I also find another bug about the caculation of the variable splitCount
> which is cause by the wrong caculation of variable finished.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11643) Read and write MOB in HBase
[
https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093744#comment-14093744
]
Hadoop QA commented on HBASE-11643:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661130/HBASE-11643-V6.diff
against trunk revision .
ATTACHMENT ID: 12661130
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestClassFinder
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10392//console
This message is automatically generated.
> Read and write MOB in HBase
> ---
>
> Key: HBASE-11643
> URL: https://issues.apache.org/jira/browse/HBASE-11643
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver, Scanners
>Reporter: Jingcheng Du
>Assignee: Jingcheng Du
> Attachments: HBASE-11643-V2.diff, HBASE-11643-V3.diff,
> HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff,
> HBase-11643.diff
>
>
> The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells
> are saved in the MemStore, and flushed to the HFiles when necessary. For MOB,
> the Cells are saved in the MemStore as well, but they're flushed to elsewhere
> out of HBase in the format of HFiles.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11660) Make WAL reader follow contract for java.io.InputStream.available()
[ https://issues.apache.org/jira/browse/HBASE-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11660: -- Status: Patch Available (was: Open) Submitting patch to see how the patch does. Would be cool if we could do w/o. I seem to remember it needed but can't remember why. > Make WAL reader follow contract for java.io.InputStream.available() > --- > > Key: HBASE-11660 > URL: https://issues.apache.org/jira/browse/HBASE-11660 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.99.0, 2.0.0, 0.98.6 >Reporter: Eric Hanson >Priority: Minor > Attachments: hbase-11660.01.patch > > > In the process of building support to running HBase on Microsoft Azure > HDInsight, I hit an issue in the HBase WAL reading process that took a lot of > time to debug. The WAL reading code depends on available() for the log > InputStream never returing 0 until end of file. This is not the same as the > contract in java.io.InputStream for available. > To prevent future grief for others that may want to port HBase onto storage > systems other than HDFS, I propose to change the HBase WAL reader so it does > not assume that EOF has been reached when available() == 0. It instead would > treat available only as described in InputStream, i.e. available() is merely > the number of bytes that could be read from the stream without blocking. That > could be 0 even before EOF. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[
https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093739#comment-14093739
]
Aleksandr Shulman commented on HBASE-11721:
---
Thanks [~misty] for trying out the tool. I am open to suggestions on how to
make the usage instructions more clear.
Regarding the error you are seeing:
Looks like the artifact that this script depends on may have been moved. curl l
is returning an html redirect (301) instead of the zip file.
Using wget, it looks like there may also be some openssl or certificate issues.
Doing a little homework on the matter, it looks like some people have hit this
issue with . Will look into this further.
{code}wget
http://cloud.github.com/downloads/tomwhite/jdiff/jdiff-1.1.1-with-incompatible-option.zip
--2014-08-11 21:42:11--
http://cloud.github.com/downloads/tomwhite/jdiff/jdiff-1.1.1-with-incompatible-option.zip
Resolving cloud.github.com (cloud.github.com)... 54.230.141.84, 54.230.143.7,
54.230.140.148, ...
Connecting to cloud.github.com (cloud.github.com)|54.230.141.84|:80...
connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location:
https://cloud.github.com/downloads/tomwhite/jdiff/jdiff-1.1.1-with-incompatible-option.zip
[following]
--2014-08-11 21:42:11--
https://cloud.github.com/downloads/tomwhite/jdiff/jdiff-1.1.1-with-incompatible-option.zip
Connecting to cloud.github.com (cloud.github.com)|54.230.141.84|:443...
connected.
OpenSSL: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
handshake failure
Unable to establish SSL connection.
{code}
> jdiff script no longer works as usage instructions indicate
> ---
>
> Key: HBASE-11721
> URL: https://issues.apache.org/jira/browse/HBASE-11721
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Reporter: Misty Stanley-Jones
>
> I pasted the command from the usage instructions embedded in the script, but
> it fails as follows:
> [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh
> https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git
> 0.94
> JDiff evaluation beginning:
> Determining if this is a local directory or a git repo.
> Looks like https://github.com/apache/hbase.git is a git repo
> Determining if this is a local directory or a git repo.
> Looks like https://github.com/MY_REPO/hbase.git is a git repo
> We are going to compare source 1 which is a git_repo and source 2, which is a
> git_repo
> 0.94
> 0.94
> JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to
> /tmp/jdiff.
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448
> Archive: jdiff-1.1.1-with-incompatible-option.zip
> End-of-central-directory signature not found. Either this file is not
> a zipfile, or it constitutes one disk of a multi-part archive. In the
> latter case the central directory and zipfile comment will be found on
> the last disk(s) of this archive.
> unzip: cannot find zipfile directory in one of
> jdiff-1.1.1-with-incompatible-option.zip or
> jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find
> jdiff-1.1.1-with-incompatible-option.zip.ZIP, period.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093736#comment-14093736 ] ramkrishna.s.vasudevan commented on HBASE-11438: Thanks [~apurtell]. > [Visibility Controller] Support UTF8 character as Visibility Labels > --- > > Key: HBASE-11438 > URL: https://issues.apache.org/jira/browse/HBASE-11438 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.4 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.6 > > Attachments: HBASE-11438_v1.patch, HBASE-11438_v2.patch, > HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, > HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, > HBASE-11438_v9.patch > > > This would be an action item that we would be addressing so that the > visibility labels could have UTF8 characters in them. Also allow the user to > use a client supplied API that allows to specify the visibility labels inside > double quotes such that UTF8 characters and cases like &, |, ! and double > quotes itself could be specified with proper escape sequence. Accumulo too > provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11639) [Visibility controller] Replicate the visibility of Cells as strings
[ https://issues.apache.org/jira/browse/HBASE-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093735#comment-14093735 ] ramkrishna.s.vasudevan commented on HBASE-11639: Actually Anoop had a patch like a POC which is not yet tested and I had some patch but with more new hooks. We need to see them once and test them but I think we could target this by the end of august. We will change the assignee based on that. Fine [~apurtell]? > [Visibility controller] Replicate the visibility of Cells as strings > > > Key: HBASE-11639 > URL: https://issues.apache.org/jira/browse/HBASE-11639 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.4 >Reporter: ramkrishna.s.vasudevan > Fix For: 0.99.0, 0.98.6 > > > This issue is aimed at persisting the visibility labels as strings in the WAL > rather than Label ordinals. This would help in replicating the label > ordinals to the replication cluster as strings directly and also that after > HBASE-11553 would help because the replication cluster could have an > implementation as string based visibility labels. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11604) Disable co-locating meta/master by default
[ https://issues.apache.org/jira/browse/HBASE-11604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093715#comment-14093715 ] stack commented on HBASE-11604: --- If on one machine -- i.e. pseudo distributed -- can't we tell if the RS if ours or that of another RS? We know our startcode? I ask because asking for second config. just to do pseudo is going to be a little surprising and makes the setup of pseudo just a little bit more complicated. Thanks Jimmy. > Disable co-locating meta/master by default > -- > > Key: HBASE-11604 > URL: https://issues.apache.org/jira/browse/HBASE-11604 > Project: HBase > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 1.0.0 > > Attachments: hbase-11604.patch, hbase-11604_v2.patch > > > To avoid possible confusing, it's better to keep the original deployment > scheme in 1.0. ZK-less region assignment is off by default in 1.0 already. We > should, by default, not assign any region to master or backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated HBASE-11685:
Attachment: HBASE-11685-trunk-v5.diff
Updates for [~apurtell]'s review
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff, HBASE-11685-trunk-v2.diff,
> HBASE-11685-trunk-v3.diff, HBASE-11685-trunk-v4.diff,
> HBASE-11685-trunk-v5.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11708) RegionSplitter incorrectly calculates splitcount
[
https://issues.apache.org/jira/browse/HBASE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093697#comment-14093697
]
louis hust commented on HBASE-11708:
After create 't1','f1'
I run the following command:
bin/hbase org.apache.hadoop.hbase.util.RegionSplitter -r -o 2 t1 UniformSplit
The behaviour is different before and after the patch:
Before this patch:
2014-08-12 11:38:39,932 DEBUG [main] util.RegionSplitter: Split Scan: 0
finished / 1 split wait / 0 reference wait
2014-08-12 11:38:44,977 DEBUG [main] util.RegionSplitter: Split Scan: 1
finished / 0 split wait / 0 reference wait
2014-08-12 11:38:44,977 DEBUG [main] util.RegionSplitter: All regions have been
successfully split!
2014-08-12 11:38:44,977 DEBUG [main] util.RegionSplitter: TOTAL TIME = 30sec
{color:red}
2014-08-12 11:38:44,977 DEBUG [main] util.RegionSplitter: Splits = 0
{color}
2014-08-12 11:38:44,977 DEBUG [main] util.RegionSplitter: Avg Time / Split =
30sec
After this patch:
2014-08-12 11:35:08,214 DEBUG [main] util.RegionSplitter: Split Scan: 0
finished / 1 split wait / 0 reference wait
2014-08-12 11:35:13,250 DEBUG [main] util.RegionSplitter: Split Scan: 1
finished / 0 split wait / 0 reference wait
2014-08-12 11:35:13,250 DEBUG [main] util.RegionSplitter: All regions have been
successfully split!
2014-08-12 11:35:13,250 DEBUG [main] util.RegionSplitter: TOTAL TIME = 30sec
{color:red}
2014-08-12 11:35:13,250 DEBUG [main] util.RegionSplitter: Splits = 1
{color}
2014-08-12 11:35:13,250 DEBUG [main] util.RegionSplitter: Avg Time / Split =
30sec
> RegionSplitter incorrectly calculates splitcount
>
>
> Key: HBASE-11708
> URL: https://issues.apache.org/jira/browse/HBASE-11708
> Project: HBase
> Issue Type: Bug
> Components: Admin, util
>Affects Versions: 0.96.2, 0.98.1
>Reporter: Sean Busbey
>Assignee: louis hust
>Priority: Minor
> Labels: beginner
> Attachments: HBASE_11708-v2.patch, HBASE_11708-v3.patch,
> HBASE_11708.patch
>
>
> From discussion on HBASE-11627:
> {quote}
> And I also find another bug about the caculation of the variable splitCount
> which is cause by the wrong caculation of variable finished.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11643) Read and write MOB in HBase
[
https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingcheng Du updated HBASE-11643:
-
Attachment: HBASE-11643-V6.diff
Update the mob directory. In order to serve the snapshot better, the mob
directory is changed to
{hbaseRootDir}/mobdir/data/{namesapce}/{tableName}/{dummyMobRegionEncodedName}/{familyName}.
In the dummy mob region, the start key is .mob, the regionid is 0.
> Read and write MOB in HBase
> ---
>
> Key: HBASE-11643
> URL: https://issues.apache.org/jira/browse/HBASE-11643
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver, Scanners
>Reporter: Jingcheng Du
>Assignee: Jingcheng Du
> Attachments: HBASE-11643-V2.diff, HBASE-11643-V3.diff,
> HBASE-11643-V4.diff, HBASE-11643-V5.diff, HBASE-11643-V6.diff,
> HBase-11643.diff
>
>
> The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells
> are saved in the MemStore, and flushed to the HFiles when necessary. For MOB,
> the Cells are saved in the MemStore as well, but they're flushed to elsewhere
> out of HBase in the format of HFiles.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11707) Using Map instead of list in FailedServers of RpcClient
[
https://issues.apache.org/jira/browse/HBASE-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093691#comment-14093691
]
Liu Shaohui commented on HBASE-11707:
-
[~nkeywal]
{quote}
The isFailedServer and addToFailedServers method may run concurently, so i use
the concurrentMap.
The two methods (isFailedServer and addToFailedServers) are synchronized.
What's the concurrency issue?
{quote}
It's my mistake. Thank you for pointing out it.
> Using Map instead of list in FailedServers of RpcClient
> ---
>
> Key: HBASE-11707
> URL: https://issues.apache.org/jira/browse/HBASE-11707
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11707-trunk-v1.diff, HBASE-11707-trunk-v2.diff
>
>
> Currently, FailedServers uses a list to record the black list of servers and
> iterate the list to check if a server is in list. It's not efficient when the
> list is very large. And the list is not thread safe for the add and iteration
> operations.
> RpcClient.java#175
> {code}
> // iterate, looking for the search entry and cleaning expired entries
> Iterator> it = failedServers.iterator();
> while (it.hasNext()) {
> Pair cur = it.next();
> if (cur.getFirst() < now) {
> it.remove();
> } else {
> if (lookup.equals(cur.getSecond())) {
> return true;
> }
> }
> {code}
> A simple change is to change this list to ConcurrentHashMap.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11707) Using Map instead of list in FailedServers of RpcClient
[
https://issues.apache.org/jira/browse/HBASE-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated HBASE-11707:
Attachment: HBASE-11707-trunk-v2.diff
Thanks for [~nkeywal]'s review.
> Using Map instead of list in FailedServers of RpcClient
> ---
>
> Key: HBASE-11707
> URL: https://issues.apache.org/jira/browse/HBASE-11707
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11707-trunk-v1.diff, HBASE-11707-trunk-v2.diff
>
>
> Currently, FailedServers uses a list to record the black list of servers and
> iterate the list to check if a server is in list. It's not efficient when the
> list is very large. And the list is not thread safe for the add and iteration
> operations.
> RpcClient.java#175
> {code}
> // iterate, looking for the search entry and cleaning expired entries
> Iterator> it = failedServers.iterator();
> while (it.hasNext()) {
> Pair cur = it.next();
> if (cur.getFirst() < now) {
> it.remove();
> } else {
> if (lookup.equals(cur.getSecond())) {
> return true;
> }
> }
> {code}
> A simple change is to change this list to ConcurrentHashMap.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11708) RegionSplitter incorrectly calculates splitcount
[
https://issues.apache.org/jira/browse/HBASE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093678#comment-14093678
]
Sean Busbey commented on HBASE-11708:
-
Yeah, I can't find anything in the extant test tools for verifying logs.
Could you attach a log snippet from before and after the patch that shows the
count being corrected?
> RegionSplitter incorrectly calculates splitcount
>
>
> Key: HBASE-11708
> URL: https://issues.apache.org/jira/browse/HBASE-11708
> Project: HBase
> Issue Type: Bug
> Components: Admin, util
>Affects Versions: 0.96.2, 0.98.1
>Reporter: Sean Busbey
>Assignee: louis hust
>Priority: Minor
> Labels: beginner
> Attachments: HBASE_11708-v2.patch, HBASE_11708-v3.patch,
> HBASE_11708.patch
>
>
> From discussion on HBASE-11627:
> {quote}
> And I also find another bug about the caculation of the variable splitCount
> which is cause by the wrong caculation of variable finished.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11708) RegionSplitter incorrectly calculates splitcount
[
https://issues.apache.org/jira/browse/HBASE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louis hust updated HBASE-11708:
---
Attachment: HBASE_11708-v3.patch
[~busbey], the test case is not proper for this issue, and i can't figure out
how to add a test case to check the splitCount value, So i discard the test
case.
> RegionSplitter incorrectly calculates splitcount
>
>
> Key: HBASE-11708
> URL: https://issues.apache.org/jira/browse/HBASE-11708
> Project: HBase
> Issue Type: Bug
> Components: Admin, util
>Affects Versions: 0.96.2, 0.98.1
>Reporter: Sean Busbey
>Assignee: louis hust
>Priority: Minor
> Labels: beginner
> Attachments: HBASE_11708-v2.patch, HBASE_11708-v3.patch,
> HBASE_11708.patch
>
>
> From discussion on HBASE-11627:
> {quote}
> And I also find another bug about the caculation of the variable splitCount
> which is cause by the wrong caculation of variable finished.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-9531) a command line (hbase shell) interface to retreive the replication metrics and show replication lag
[
https://issues.apache.org/jira/browse/HBASE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Demai Ni updated HBASE-9531:
Attachment: HBASE-9531-master-v3.patch
upload v3 patch.
[~apurtell], can you please take a look again? Originally, I added the
ReplicationLoadSink and ReplicationLoadSource inside ReplicationLoad.java.
However, it turns out that ProtobufUtil.java can't import ReplicationLoad
directly as it is under hbase-server. So I just put the two new files under
hbase-client.
> a command line (hbase shell) interface to retreive the replication metrics
> and show replication lag
> ---
>
> Key: HBASE-9531
> URL: https://issues.apache.org/jira/browse/HBASE-9531
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Affects Versions: 0.99.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 0.99.0, 0.98.6
>
> Attachments: HBASE-9531-master-v1.patch, HBASE-9531-master-v1.patch,
> HBASE-9531-master-v1.patch, HBASE-9531-master-v2.patch,
> HBASE-9531-master-v3.patch, HBASE-9531-trunk-v0.patch,
> HBASE-9531-trunk-v0.patch
>
>
> This jira is to provide a command line (hbase shell) interface to retreive
> the replication metrics info such as:ageOfLastShippedOp,
> timeStampsOfLastShippedOp, sizeOfLogQueue ageOfLastAppliedOp, and
> timeStampsOfLastAppliedOp. And also to provide a point of time info of the
> lag of replication(source only)
> Understand that hbase is using Hadoop
> metrics(http://hbase.apache.org/metrics.html), which is a common way to
> monitor metric info. This Jira is to serve as a light-weight client
> interface, comparing to a completed(certainly better, but heavier)GUI
> monitoring package. I made the code works on 0.94.9 now, and like to use this
> jira to get opinions about whether the feature is valuable to other
> users/workshop. If so, I will build a trunk patch.
> All inputs are greatly appreciated. Thank you!
> The overall design is to reuse the existing logic which supports hbase shell
> command 'status', and invent a new module, called ReplicationLoad. In
> HRegionServer.buildServerLoad() , use the local replication service objects
> to get their loads which could be wrapped in a ReplicationLoad object and
> then simply pass it to the ServerLoad. In ReplicationSourceMetrics and
> ReplicationSinkMetrics, a few getters and setters will be created, and ask
> Replication to build a "ReplicationLoad". (many thanks to Jean-Daniel for
> his kindly suggestions through dev email list)
> the replication lag will be calculated for source only, and use this formula:
> {code:title=Replication lag|borderStyle=solid}
> if sizeOfLogQueue != 0 then max(ageOfLastShippedOp, (current time -
> timeStampsOfLastShippedOp)) //err on the large side
> else if (current time - timeStampsOfLastShippedOp) < 2*
> ageOfLastShippedOp then lag = ageOfLastShippedOp // last shipped happen
> recently
> else lag = 0 // last shipped may happens last night, so NO real lag
> although ageOfLastShippedOp is non-zero
> {code}
> External will look something like:
> {code:title=status 'replication'|borderStyle=solid}
> hbase(main):001:0> status 'replication'
> version 0.94.9
> 3 live servers
> Â Â Â hdtest017.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:49:48 PDT 2013
> Â Â Â SINKÂ :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:48:48 PDT 2013
> Â Â Â hdtest018.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> Â Â Â SINKÂ :AgeOfLastAppliedOp=14, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:50:59 PDT 2013
> Â Â Â hdtest015.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> Â Â Â SINKÂ :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:48:48 PDT 2013
> hbase(main):002:0> status 'replication','source'
> version 0.94.9
> 3 live servers
> Â Â Â hdtest017.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=14, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:49:48 PDT 2013
> Â Â Â hdtest018.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> Â Â Â hdtest015.svl.ibm.com:
> Â Â Â SOURCE:PeerID=1, ageOfLastShippedOp=0, sizeOfLogQueue=0,
> timeStampsOfLastShippedOp=Wed Sep 04 14:48:48 PDT 2013
> hbase(main):003:0> status 'replication','sink'
> version 0.94.9
> 3 live servers
> Â Â Â hdtest017.svl.ibm.com:
> Â Â Â SINKÂ :AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Wed Sep 04
> 14:48:48 PDT 2013
> Â Â Â hdtes
[jira] [Created] (HBASE-11722) Document new shortcut commands introduced by HBASE-11649
Misty Stanley-Jones created HBASE-11722: --- Summary: Document new shortcut commands introduced by HBASE-11649 Key: HBASE-11722 URL: https://issues.apache.org/jira/browse/HBASE-11722 Project: HBase Issue Type: Sub-task Components: documentation Affects Versions: 0.98.4 Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11656) Document how to script snapshots
[
https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093664#comment-14093664
]
Sean Busbey commented on HBASE-11656:
-
{quote}
+ connection, which would look like a failure to the client. The only
way to definitively
+ know that a snapshot succeeded is to check whether it exists.
Following is a simple
{quote}
This should be "The only way to definitively know that a snapshot failed is to
check wether it exists."
It would be a bug for the command invocation to exit with status 0 and for the
snapshot to have failed.
> Document how to script snapshots
>
>
> Key: HBASE-11656
> URL: https://issues.apache.org/jira/browse/HBASE-11656
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, shell
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11656.patch, HBASE-11656.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11656) Document how to script snapshots
[
https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093663#comment-14093663
]
Sean Busbey commented on HBASE-11656:
-
{quote}
bq. If the snapshot is successful, the command will return 0. Otherwise, it
will return a non-zero value.
That is 100% not correct. If the client hit the operation timeout or lose the
connection with the server you'll get a failure (non-zero value), which you are
not able to interpret. The snapshot may be created or may not be created.
{quote}
Unknown state after timeout/connection errors are a problem with a bunch of the
shell operations. Do we need to have a specified return code for them so that
the docs can call it out as a special case before adding examples at all under
HBASE-11655?
> Document how to script snapshots
>
>
> Key: HBASE-11656
> URL: https://issues.apache.org/jira/browse/HBASE-11656
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, shell
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11656.patch, HBASE-11656.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11660) Make WAL reader follow contract for java.io.InputStream.available()
[ https://issues.apache.org/jira/browse/HBASE-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093651#comment-14093651 ] Enis Soztutar commented on HBASE-11660: --- Agreed with Eric and Jeff that we should do a sweep for removing dependency on available() since it does not match the documented java semantics. Great find Eric! I'll try to take a shot if I find time. > Make WAL reader follow contract for java.io.InputStream.available() > --- > > Key: HBASE-11660 > URL: https://issues.apache.org/jira/browse/HBASE-11660 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.99.0, 2.0.0, 0.98.6 >Reporter: Eric Hanson >Priority: Minor > Attachments: hbase-11660.01.patch > > > In the process of building support to running HBase on Microsoft Azure > HDInsight, I hit an issue in the HBase WAL reading process that took a lot of > time to debug. The WAL reading code depends on available() for the log > InputStream never returing 0 until end of file. This is not the same as the > contract in java.io.InputStream for available. > To prevent future grief for others that may want to port HBase onto storage > systems other than HDFS, I propose to change the HBase WAL reader so it does > not assume that EOF has been reached when available() == 0. It instead would > treat available only as described in InputStream, i.e. available() is merely > the number of bytes that could be read from the stream without blocking. That > could be 0 even before EOF. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11656) Document how to script snapshots
[
https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093647#comment-14093647
]
Hadoop QA commented on HBASE-11656:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661092/HBASE-11656.patch
against trunk revision .
ATTACHMENT ID: 12661092
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.regionserver.TestRegionFavoredNodes
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicasClient.testSmallScanWithReplicas(TestReplicasClient.java:536)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10389//console
This message is automatically generated.
> Document how to script snapshots
>
>
> Key: HBASE-11656
> URL: https://issues.apache.org/jira/browse/HBASE-11656
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, shell
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11656.patch, HBASE-11656.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11655) Document how to use Bash with HBase Shell
[
https://issues.apache.org/jira/browse/HBASE-11655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093637#comment-14093637
]
Hadoop QA commented on HBASE-11655:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661095/HBASE-11655.patch
against trunk revision .
ATTACHMENT ID: 12661095
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10388//console
This message is automatically generated.
> Document how to use Bash with HBase Shell
> -
>
> Key: HBASE-11655
> URL: https://issues.apache.org/jira/browse/HBASE-11655
> Project: HBase
> Issue Type: Task
> Components: documentation, shell
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11655.patch, HBASE-11655.patch, HBASE-11655.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11716) LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
[
https://issues.apache.org/jira/browse/HBASE-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093610#comment-14093610
]
Hudson commented on HBASE-11716:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #418 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/418/])
HBASE-11716 LoadTestDataGeneratorWithVisibilityLabels should handle Delete
mutations (Anoop Sam John) (apurtell: rev
93390aca70a2780fd1a017a8db28ad9b01a808fc)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/LoadTestDataGeneratorWithVisibilityLabels.java
> LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
>
>
> Key: HBASE-11716
> URL: https://issues.apache.org/jira/browse/HBASE-11716
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.5
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Labels: IntegrationTest
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11716.patch
>
>
> {code}
> public Mutation beforeMutate(long rowkeyBase, Mutation m) throws
> IOException {
> if (!(m instanceof Delete)) {
> m.setCellVisibility(new CellVisibility(this.visibilityExps[(int)
> rowkeyBase
> % this.visibilityExps.length]));
> }
> return m;
> }
> {code}
> Now we have delete CellVisibility matching and so we should add
> CellVisibility for Mutations also. Just removing the if check part will solve
> the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: HBASE_11699.patch > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093602#comment-14093602
]
Enis Soztutar commented on HBASE-11657:
---
Carter, I really like this jira. Let me take a deeper look at the patch and the
rest of the comments.
> Put HTable region methods in an interface
> -
>
> Key: HBASE-11657
> URL: https://issues.apache.org/jira/browse/HBASE-11657
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Carter
>Assignee: Carter
> Fix For: 0.99.0
>
> Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
> HBASE_11657_v3.patch, HBASE_11657_v3.patch, HBASE_11657_v4.patch
>
>
> Most of the HTable methods are now abstracted by HTableInterface, with the
> notable exception of the following methods that pertain to region metadata:
> {code}
> HRegionLocation getRegionLocation(final String row)
> HRegionLocation getRegionLocation(final byte [] row)
> HRegionLocation getRegionLocation(final byte [] row, boolean reload)
> byte [][] getStartKeys()
> byte[][] getEndKeys()
> Pair getStartEndKeys()
> void clearRegionCache()
> {code}
> and a default scope method which maybe should be bundled with the others:
> {code}
> List listRegionLocations()
> {code}
> Since the consensus seems to be that these would muddy HTableInterface with
> non-core functionality, where should it go? MapReduce looks up the region
> boundaries, so it needs to be exposed somewhere.
> Let me throw out a straw man to start the conversation. I propose:
> {code}
> org.apache.hadoop.hbase.client.HRegionInterface
> {code}
> Have HTable implement this interface. Also add these methods to HConnection:
> {code}
> HRegionInterface getTableRegion(TableName tableName)
> HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
> {code}
> [~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: (was: HBASE_11699.patch) > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Status: Open (was: Patch Available) > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Status: Patch Available (was: Open) > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11702) Better introspection of long running compactions
[
https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093599#comment-14093599
]
Hadoop QA commented on HBASE-11702:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661087/HBASE-11702.patch
against trunk revision .
ATTACHMENT ID: 12661087
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10387//console
This message is automatically generated.
> Better introspection of long running compactions
>
>
> Key: HBASE-11702
> URL: https://issues.apache.org/jira/browse/HBASE-11702
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11702.patch
>
>
> For better introspection of long running compactions, periodically print
> compaction progress for a file at DEBUG level (thread name, file path, total
> compacted KVs, total compacted bytes, completion percent, rate).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093590#comment-14093590 ] Misty Stanley-Jones commented on HBASE-11721: - OK so I realized that I didn't read closely enough and replace the MY_REPO part so I tried several other variants: bash ./jdiffHBasePublicAPI.sh https://git.apache.org/hbase.git 0.94 https://git.apache.org/hbase.git 0.96 bash ./jdiffHBasePublicAPI.sh https://git-wip-us.apache.org/repos/asf?p=hbase.git 0.94 https://git-wip-us.apache.org/repos/asf?p=hbase.git 0.96 I get the same result: JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://git-wip-us.apache.org/repos/asf?p=hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://git-wip-us.apache.org/repos/asf?p=hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.96 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0264 0 --:--:-- --:--:-- --:--:-- 264 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. > jdiff script no longer works as usage instructions indicate > --- > > Key: HBASE-11721 > URL: https://issues.apache.org/jira/browse/HBASE-11721 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Misty Stanley-Jones > > I pasted the command from the usage instructions embedded in the script, but > it fails as follows: > [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh > https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git > 0.94 > JDiff evaluation beginning: > Determining if this is a local directory or a git repo. > Looks like https://github.com/apache/hbase.git is a git repo > Determining if this is a local directory or a git repo. > Looks like https://github.com/MY_REPO/hbase.git is a git repo > We are going to compare source 1 which is a git_repo and source 2, which is a > git_repo > 0.94 > 0.94 > JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to > /tmp/jdiff. > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 > Archive: jdiff-1.1.1-with-incompatible-option.zip > End-of-central-directory signature not found. Either this file is not > a zipfile, or it constitutes one disk of a multi-part archive. In the > latter case the central directory and zipfile comment will be found on > the last disk(s) of this archive. > unzip: cannot find zipfile directory in one of > jdiff-1.1.1-with-incompatible-option.zip or > jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find > jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10247) Client promises about timestamps
[ https://issues.apache.org/jira/browse/HBASE-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-10247: -- Attachment: 10247-do-not-try-may-eat-your-first-born-v2.txt Updated patch, again just parking. Disallows Put/Delete according to TSPOLICY, adds tests for all policies. Next: * Appends, Increments * Restrict alters * Disallow TTL with no wall clock type TSPOLICY > Client promises about timestamps > > > Key: HBASE-10247 > URL: https://issues.apache.org/jira/browse/HBASE-10247 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 0.99.0, 2.0.0 > > Attachments: 10247-do-not-try-may-eat-your-first-born-v2.txt, > 10247.txt > > > This is to start a discussion about timestamp promises declared per table of > CF. > For example if a client promises only monotonically increasing timestamps (or > no custom set timestamps) and VERSIONS=1, we can aggressively and easily > remove old versions of the same row/fam/col from the memstore before we > flush, just by supplying a comparator that ignores the timestamp (i.e. two KV > just differing by TS would be considered equal). > That would increase the performance of counters significantly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10247) Client promises about timestamps
[ https://issues.apache.org/jira/browse/HBASE-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093578#comment-14093578 ] Lars Hofhansl commented on HBASE-10247: --- Here's an interesting question: What should happen with TS specific deletes when the client does not set the ts? If we allow to data deletes into the future, we'll have won nothing. If we disallow client set timestamps for deletes, then we can no longer target a specific version for delete (for deleteColumn), nor target only some versions for delete (deleteColumns and deleteFamily). So if the server sets the TS or SeqId, we can no longer delete specific version, not can we delete only some columns in the past. I suppose we could allow timestamp that are strictly older than the next timestamp the server would hand out...? > Client promises about timestamps > > > Key: HBASE-10247 > URL: https://issues.apache.org/jira/browse/HBASE-10247 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 0.99.0, 2.0.0 > > Attachments: 10247.txt > > > This is to start a discussion about timestamp promises declared per table of > CF. > For example if a client promises only monotonically increasing timestamps (or > no custom set timestamps) and VERSIONS=1, we can aggressively and easily > remove old versions of the same row/fam/col from the memstore before we > flush, just by supplying a comparator that ignores the timestamp (i.e. two KV > just differing by TS would be considered equal). > That would increase the performance of counters significantly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093580#comment-14093580 ] Misty Stanley-Jones commented on HBASE-11721: - Script was developed by [~aleksshulman] > jdiff script no longer works as usage instructions indicate > --- > > Key: HBASE-11721 > URL: https://issues.apache.org/jira/browse/HBASE-11721 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Misty Stanley-Jones > > I pasted the command from the usage instructions embedded in the script, but > it fails as follows: > [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh > https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git > 0.94 > JDiff evaluation beginning: > Determining if this is a local directory or a git repo. > Looks like https://github.com/apache/hbase.git is a git repo > Determining if this is a local directory or a git repo. > Looks like https://github.com/MY_REPO/hbase.git is a git repo > We are going to compare source 1 which is a git_repo and source 2, which is a > git_repo > 0.94 > 0.94 > JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to > /tmp/jdiff. > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 > Archive: jdiff-1.1.1-with-incompatible-option.zip > End-of-central-directory signature not found. Either this file is not > a zipfile, or it constitutes one disk of a multi-part archive. In the > latter case the central directory and zipfile comment will be found on > the last disk(s) of this archive. > unzip: cannot find zipfile directory in one of > jdiff-1.1.1-with-incompatible-option.zip or > jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find > jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11721) jdiff script no longer works as usage instructions indicate
Misty Stanley-Jones created HBASE-11721: --- Summary: jdiff script no longer works as usage instructions indicate Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11467) New impl of Registry interface not using ZK + new RPCs on master protocol
[ https://issues.apache.org/jira/browse/HBASE-11467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093566#comment-14093566 ] Gary Helmling commented on HBASE-11467: --- [~stack] The client needs to know the ClusterId in order to select the token to use for token authentication from the bundle of credentials that it has access to. I believe when we first implemented this we discussed using the zk quorum string instead, but we settled on using a globally unique cluster ID as a cleaner solution, due to difficulties in canonicalizing the quorum string (short vs. fully qualified hostnames, ordering, whether or not client port is present...), and of course the zk quorum could change. It might be possible to have the server provide the cluster ID as part of the initial SASL negotiation, but I'm not sure how difficult this would be. Currently, if a token is found, we negotiate DIGEST-MD5 with SASL, otherwise pass through to GSSAPI. So having the server return a cluster ID in negotiation would require a step before that in order to determine whether we have a token or not. I agree that forcing the client to add the cluster ID in configuration somewhere is not a great option, but I haven't looked at the patch yet to see what it is doing. I will take a look at the details. > New impl of Registry interface not using ZK + new RPCs on master protocol > - > > Key: HBASE-11467 > URL: https://issues.apache.org/jira/browse/HBASE-11467 > Project: HBase > Issue Type: Sub-task > Components: Client, Consensus, Zookeeper >Affects Versions: 2.0.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Fix For: 2.0.0 > > Attachments: HBASE-11467.patch, HBASE-11467.patch > > > Currently there' only one implementation of Registry interface, which is > using ZK to get info about meta. Need to create implementation which will be > using RPC calls to master the client is connected to. > Review of early version of patch is here: https://reviews.apache.org/r/24296/ -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11711) In some environment HBase MiniCluster fails to load because Master info port clobbering
[ https://issues.apache.org/jira/browse/HBASE-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-11711: Description: Currently TestZooKeeper attempts to launch to masters and the info ports collide. This disables the info port for this test. 2014-08-11 17:33:36,863 WARN [main] log.Slf4jLog(76): failed Server@d919544: java.net.BindException: Address already in use 2014-08-11 17:33:36,864 ERROR [main] hbase.MiniHBaseCluster(230): Error starting cluster java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMasterAddress already in use at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:147) at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:214) at org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152) at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214) at org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:896) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:865) at org.apache.hadoop.hbase.TestZooKeeper.setUp(TestZooKeeper.java:108) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.junit.runners.Suite.runChild(Suite.java:127) at org.junit.runners.Suite.runChild(Suite.java:26) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67) Caused by: java.io.IOException: Failed to start redirecting jetty server at org.apache.hadoop.hbase.master.HMaster.putUpJettyServer(HMaster.java:330) at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:304) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:142) ... 39 more Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:414) at sun.nio.ch.Net.bind(Net.java:406) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:21
[jira] [Commented] (HBASE-11711) In some environment HBase MiniCluster fails to load because Master info port clobbering
[ https://issues.apache.org/jira/browse/HBASE-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093564#comment-14093564 ] Alex Newman commented on HBASE-11711: - I wonder how this ever works. Should the info port be spawing at all? > In some environment HBase MiniCluster fails to load because Master info port > clobbering > --- > > Key: HBASE-11711 > URL: https://issues.apache.org/jira/browse/HBASE-11711 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 2.0.0 > > Attachments: HBASE-11711-v1.patch > > > Currently TestZooKeeper attempts to launch to masters and the info ports > collide. This disables the info port for this test. > 2014-08-11 17:33:36,863 WARN [main] log.Slf4jLog(76): failed Server@d919544: > java.net.BindException: Address already in use > 2014-08-11 17:33:36,864 ERROR [main] hbase.MiniHBaseCluster(230): Error > starting cluster > java.lang.RuntimeException: Failed construction of Master: class > org.apache.hadoop.hbase.master.HMasterAddress already in use > at > org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:147) > at > org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:214) > at > org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152) > at > org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214) > at > org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:896) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:865) > at org.apache.hadoop.hbase.TestZooKeeper.setUp(TestZooKeeper.java:108) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at org.junit.runners.Suite.runChild(Suite.java:127) > at org.junit.runners.Suite.runChild(Suite.java:26) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) >at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at org.junit.runner.JUnitCore.run(JUnitCore.java:160) > at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74) > at > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211) > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67) > Caused by: java.io.IOException: Failed to start redirecting jetty server > at > org.apache.hadoop.hbase.master.HMaster.putUpJettyServer(HMaster.java:330) > at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:304) > at sun.re
[jira] [Commented] (HBASE-11617) incorrect AgeOfLastAppliedOp and AgeOfLastShippedOp in replication Metrics when no new replication OP
[
https://issues.apache.org/jira/browse/HBASE-11617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093561#comment-14093561
]
Andrew Purtell commented on HBASE-11617:
Any chance for a look [~lhofhansl]?
> incorrect AgeOfLastAppliedOp and AgeOfLastShippedOp in replication Metrics
> when no new replication OP
> --
>
> Key: HBASE-11617
> URL: https://issues.apache.org/jira/browse/HBASE-11617
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.98.2
>Reporter: Demai Ni
>Assignee: Demai Ni
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11617-master-v1.patch
>
>
> AgeOfLastAppliedOp in MetricsSink.java is to indicate the time an edit sat in
> the 'replication queue' before it got replicated(aka applied)
> {code}
> /**
>* Set the age of the last applied operation
>*
>* @param timestamp The timestamp of the last operation applied.
>* @return the age that was set
>*/
> public long setAgeOfLastAppliedOp(long timestamp) {
> lastTimestampForAge = timestamp;
> long age = System.currentTimeMillis() - lastTimestampForAge;
> rms.setGauge(SINK_AGE_OF_LAST_APPLIED_OP, age);
> return age;
> }
> {code}
> In the following scenario:
> 1) at 7:00am a sink op is applied, and the SINK_AGE_OF_LAST_APPLIED_OP is
> set for example 100ms;
> 2) and then NO new Sink op occur.
> 3) when a refreshAgeOfLastAppliedOp() is invoked at 8:00am. Instead of
> return the 100ms, the AgeOfLastAppliedOp become 1hour + 100ms,
> It was because that refreshAgeOfLastAppliedOp() get invoked periodically by
> getStats().
> proposed fix:
> {code}
> ---
> hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSink.java
> +++
> hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSink.java
> @@ -35,6 +35,7 @@ public class MetricsSink {
>
>private MetricsReplicationSource rms;
>private long lastTimestampForAge = System.currentTimeMillis();
> + private long age = 0;
>
>public MetricsSink() {
> rms =
> CompatibilitySingletonFactory.getInstance(MetricsReplicationSource.class);
> @@ -47,8 +48,12 @@ public class MetricsSink {
> * @return the age that was set
> */
>public long setAgeOfLastAppliedOp(long timestamp) {
> -lastTimestampForAge = timestamp;
> -long age = System.currentTimeMillis() - lastTimestampForAge;
> +if (lastTimestampForAge != timestamp) {
> + lastTimestampForAge = timestamp;
> + this.age = System.currentTimeMillis() - lastTimestampForAge;
> +} else {
> + this.age = 0;
> +}
> rms.setGauge(SINK_AGE_OF_LAST_APPLIED_OP, age);
> return age;
>}
> {code}
> detail discussion in [dev@hbase |
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201407.mbox/%3CCAOEq2C5BKMXAM2Fv4LGVb_Ktek-Pm%3DhjOi33gSHX-2qHqAou6w%40mail.gmail.com%3E
> ]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11639) [Visibility controller] Replicate the visibility of Cells as strings
[ https://issues.apache.org/jira/browse/HBASE-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093559#comment-14093559 ] Andrew Purtell commented on HBASE-11639: This issue doesn't have an assignee. Does it have a shot at being ready at end of August for .6? Otherwise let's move it out. > [Visibility controller] Replicate the visibility of Cells as strings > > > Key: HBASE-11639 > URL: https://issues.apache.org/jira/browse/HBASE-11639 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.4 >Reporter: ramkrishna.s.vasudevan > Fix For: 0.99.0, 0.98.6 > > > This issue is aimed at persisting the visibility labels as strings in the WAL > rather than Label ordinals. This would help in replicating the label > ordinals to the replication cluster as strings directly and also that after > HBASE-11553 would help because the replication cluster could have an > implementation as string based visibility labels. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10247) Client promises about timestamps
[ https://issues.apache.org/jira/browse/HBASE-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10247: -- Fix Version/s: 2.0.0 0.99.0 > Client promises about timestamps > > > Key: HBASE-10247 > URL: https://issues.apache.org/jira/browse/HBASE-10247 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 0.99.0, 2.0.0 > > Attachments: 10247.txt > > > This is to start a discussion about timestamp promises declared per table of > CF. > For example if a client promises only monotonically increasing timestamps (or > no custom set timestamps) and VERSIONS=1, we can aggressively and easily > remove old versions of the same row/fam/col from the memstore before we > flush, just by supplying a comparator that ignores the timestamp (i.e. two KV > just differing by TS would be considered equal). > That would increase the performance of counters significantly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093553#comment-14093553 ] Andrew Purtell edited comment on HBASE-11699 at 8/12/14 12:29 AM: -- In the 'master' branch, we no longer have a separate HMaster process. One of the regionservers will be elected to also handle the HMaster function. How do you envision a file based exclusion list working in that type of environment? was (Author: apurtell): In the master branch, we no longer have a separate master process. One of the regionservers will be elected to also handle the master function. How do you envision a file based exclusion list working in that type of environment? > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11716) LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
[
https://issues.apache.org/jira/browse/HBASE-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093551#comment-14093551
]
Hudson commented on HBASE-11716:
SUCCESS: Integrated in HBase-0.98 #444 (See
[https://builds.apache.org/job/HBase-0.98/444/])
HBASE-11716 LoadTestDataGeneratorWithVisibilityLabels should handle Delete
mutations (Anoop Sam John) (apurtell: rev
93390aca70a2780fd1a017a8db28ad9b01a808fc)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/LoadTestDataGeneratorWithVisibilityLabels.java
> LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
>
>
> Key: HBASE-11716
> URL: https://issues.apache.org/jira/browse/HBASE-11716
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.5
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Labels: IntegrationTest
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11716.patch
>
>
> {code}
> public Mutation beforeMutate(long rowkeyBase, Mutation m) throws
> IOException {
> if (!(m instanceof Delete)) {
> m.setCellVisibility(new CellVisibility(this.visibilityExps[(int)
> rowkeyBase
> % this.visibilityExps.length]));
> }
> return m;
> }
> {code}
> Now we have delete CellVisibility matching and so we should add
> CellVisibility for Mutations also. Just removing the if check part will solve
> the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093553#comment-14093553 ] Andrew Purtell commented on HBASE-11699: In the master branch, we no longer have a separate master process. One of the regionservers will be elected to also handle the master function. How do you envision a file based exclusion list working in that type of environment? > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11527) Cluster free memory limit check should consider L2 block cache size also when L2 cache is onheap.
[ https://issues.apache.org/jira/browse/HBASE-11527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093549#comment-14093549 ] Enis Soztutar commented on HBASE-11527: --- late +1 for branch-1. Thanks Anoop, Stack. > Cluster free memory limit check should consider L2 block cache size also when > L2 cache is onheap. > - > > Key: HBASE-11527 > URL: https://issues.apache.org/jira/browse/HBASE-11527 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11527.patch, HBASE-11527_0.99.patch, > HBASE-11527_addendum.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11333) Remove deprecated class MetaMigrationConvertingToPB
[ https://issues.apache.org/jira/browse/HBASE-11333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093541#comment-14093541 ] Enis Soztutar commented on HBASE-11333: --- late +1 for master. Let's not commit this in branch-1. > Remove deprecated class MetaMigrationConvertingToPB > --- > > Key: HBASE-11333 > URL: https://issues.apache.org/jira/browse/HBASE-11333 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.99.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Trivial > Fix For: 2.0.0 > > Attachments: 11333v2.txt, 11333v3.txt, HBASE-11333.patch > > > MetaMigrationConvertingToPB is marked deprecated and to be deleted next major > release after 0.96. Is that the time? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11651) Add conf which disables MetaMigrationConvertingToPB check (for experts only)
[ https://issues.apache.org/jira/browse/HBASE-11651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093540#comment-14093540 ] Enis Soztutar commented on HBASE-11651: --- late +1. > Add conf which disables MetaMigrationConvertingToPB check (for experts only) > > > Key: HBASE-11651 > URL: https://issues.apache.org/jira/browse/HBASE-11651 > Project: HBase > Issue Type: Sub-task > Components: master >Reporter: stack >Assignee: stack > Fix For: 0.99.0, 0.98.5 > > Attachments: 11651.txt > > > Up in parent issue, Virag wants to avoid expensive scan on startup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11716) LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
[
https://issues.apache.org/jira/browse/HBASE-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093537#comment-14093537
]
Hudson commented on HBASE-11716:
SUCCESS: Integrated in HBase-TRUNK #5390 (See
[https://builds.apache.org/job/HBase-TRUNK/5390/])
HBASE-11716 LoadTestDataGeneratorWithVisibilityLabels should handle Delete
mutations (Anoop Sam John) (apurtell: rev
d8a262996b43f9ffa9c641b1a0b0b2cd08677bc3)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/LoadTestDataGeneratorWithVisibilityLabels.java
> LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
>
>
> Key: HBASE-11716
> URL: https://issues.apache.org/jira/browse/HBASE-11716
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.5
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Labels: IntegrationTest
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11716.patch
>
>
> {code}
> public Mutation beforeMutate(long rowkeyBase, Mutation m) throws
> IOException {
> if (!(m instanceof Delete)) {
> m.setCellVisibility(new CellVisibility(this.visibilityExps[(int)
> rowkeyBase
> % this.visibilityExps.length]));
> }
> return m;
> }
> {code}
> Now we have delete CellVisibility matching and so we should add
> CellVisibility for Mutations also. Just removing the if check part will solve
> the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11655) Document how to use Bash with HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-11655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11655: Attachment: HBASE-11655.patch Clarified that a non-zero exit does not necessarily mean failure and the only way to know is to check. > Document how to use Bash with HBase Shell > - > > Key: HBASE-11655 > URL: https://issues.apache.org/jira/browse/HBASE-11655 > Project: HBase > Issue Type: Task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11655.patch, HBASE-11655.patch, HBASE-11655.txt > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093521#comment-14093521
]
Andrew Purtell commented on HBASE-11685:
> is the earlier logic just the current codebase?
What I mean by that is this:
{code}
void decCount() {
if (refCount > 0) {
--refCount;
}
}
{code}
This code does not allow the reference count to go below 0, which is what
isZeroReference expects. The latest patch allows the reference count to go
below 0 without changing isZeroReference or its callers to handle cases where
the refcount is negative.
> When the count goes negative, we should print warn log or throw an exception.
Throwing an exception after insuring the connection is cleaned up is good, that
would address the concern here.
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff, HBASE-11685-trunk-v2.diff,
> HBASE-11685-trunk-v3.diff, HBASE-11685-trunk-v4.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11716) LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
[
https://issues.apache.org/jira/browse/HBASE-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093522#comment-14093522
]
Hudson commented on HBASE-11716:
FAILURE: Integrated in HBase-1.0 #94 (See
[https://builds.apache.org/job/HBase-1.0/94/])
HBASE-11716 LoadTestDataGeneratorWithVisibilityLabels should handle Delete
mutations (Anoop Sam John) (apurtell: rev
cc5024e6fce985ea3266f22724942886870b042b)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/LoadTestDataGeneratorWithVisibilityLabels.java
> LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
>
>
> Key: HBASE-11716
> URL: https://issues.apache.org/jira/browse/HBASE-11716
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.5
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Labels: IntegrationTest
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11716.patch
>
>
> {code}
> public Mutation beforeMutate(long rowkeyBase, Mutation m) throws
> IOException {
> if (!(m instanceof Delete)) {
> m.setCellVisibility(new CellVisibility(this.visibilityExps[(int)
> rowkeyBase
> % this.visibilityExps.length]));
> }
> return m;
> }
> {code}
> Now we have delete CellVisibility matching and so we should add
> CellVisibility for Mutations also. Just removing the if check part will solve
> the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11567) Write bulk load COMMIT events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093520#comment-14093520 ] Enis Soztutar commented on HBASE-11567: --- After the bulk load files are committed to the region, we will call writeBulkLoadMarkerAndSync(). But if that fails with IOException or smt, then we should abort the region server similar to what we do for exception handling in flush event markers in HBASE-11511. > Write bulk load COMMIT events to WAL > > > Key: HBASE-11567 > URL: https://issues.apache.org/jira/browse/HBASE-11567 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alex Newman > Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch > > > Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and > region open/close (HBASE-11512) , we should persist bulk load events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11656) Document how to script snapshots
[ https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11656: Attachment: HBASE-11656.patch > Document how to script snapshots > > > Key: HBASE-11656 > URL: https://issues.apache.org/jira/browse/HBASE-11656 > Project: HBase > Issue Type: Sub-task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11656.patch, HBASE-11656.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11656) Document how to script snapshots
[ https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11656: Attachment: (was: HBASE-11656.patch) > Document how to script snapshots > > > Key: HBASE-11656 > URL: https://issues.apache.org/jira/browse/HBASE-11656 > Project: HBase > Issue Type: Sub-task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11656.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11656) Document how to script snapshots
[ https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11656: Attachment: HBASE-11656.patch Clarified that you can't guarantee that a non-zero status means failure. > Document how to script snapshots > > > Key: HBASE-11656 > URL: https://issues.apache.org/jira/browse/HBASE-11656 > Project: HBase > Issue Type: Sub-task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11656.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11699) Region servers exclusion list to HMaster.
[
https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093514#comment-14093514
]
Hadoop QA commented on HBASE-11699:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661026/HBASE_11699.patch
against trunk revision .
ATTACHMENT ID: 12661026
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ * rpc
RefreshIncludeExcludeRSConfig(.RefreshIncludeExcludeRSConfigRequest) returns
(.RefreshIncludeExcludeRSConfigResponse);
+ * rpc
GetExcludedRegionsServers(.GetExcludedRegionsServersRequest) returns
(.GetExcludedRegionsServersResponse);
+ * rpc GetIncludedRegionServers(.GetIncludedRegionServersRequest)
returns (.GetIncludedRegionServersResponse);
+ * rpc
RefreshIncludeExcludeRSConfig(.RefreshIncludeExcludeRSConfigRequest) returns
(.RefreshIncludeExcludeRSConfigResponse);
+ * rpc GetExcludedRegionsServers(.GetExcludedRegionsServersRequest)
returns (.GetExcludedRegionsServersResponse);
+ * rpc GetIncludedRegionServers(.GetIncludedRegionServersRequest)
returns (.GetIncludedRegionServersResponse);
+ if
(!cluster.getMaster().getServerManager().getOnlineServersList().contains(serverName))
break;
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestCheckTestClasses
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10386//console
This message is automatically generated.
> Region servers exclusion list to HMaster.
> -
>
> Key: HBASE-11699
> URL: https://issues.apache.org/jira/browse/HBASE-11699
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Client, regionserver, Zookeeper
>Affects Versions: 0.98.3
>Reporter: Gomathivinayagam Muthuvinayagam
>Priority: Minor
> Labels: patch
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE_11699.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Currently HBase does not support adding set of region servers to be in the
> exclusion list. So that administrators can prevent accidental startups of
> some region servers to join the cluster. There was initially some work done,
> and it is available in https://issues.apache.org/jira/browse/HBASE-38
[jira] [Commented] (HBASE-11656) Document how to script snapshots
[ https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093512#comment-14093512 ] Misty Stanley-Jones commented on HBASE-11656: - It was requested by [~j...@cloudera.com]. I will update it to talk about ambiguous success/failure status. > Document how to script snapshots > > > Key: HBASE-11656 > URL: https://issues.apache.org/jira/browse/HBASE-11656 > Project: HBase > Issue Type: Sub-task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11656.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093500#comment-14093500 ] Andrew Purtell commented on HBASE-11702: I decided not to export compacted KVs and total compacting KVs as Hadoop metrics as these counts are maintained per compactor and we might have more than one compactor active concurrently. Not worth the effort of making RS-wide metrics out of that. > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11702: --- Description: For better introspection of long running compactions, periodically print compaction progress for a file at DEBUG level (thread name, file path, total compacted KVs, total compacted bytes, completion percent, rate). (was: For better introspection of long running compactions, periodically print compaction progress for a file at DEBUG level (thread name, file path, total compacted KVs, total compacted bytes, completion percent, rate) and expose regionserver wide compaction metrics (KVs compacted and bytes compacted) as Hadoop value metrics.) > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11702) Better introspection of long running compactions
[
https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093494#comment-14093494
]
Andrew Purtell edited comment on HBASE-11702 at 8/11/14 11:25 PM:
--
Attached patch produces output like:
{noformat}
2014-08-11 16:20:59,739 DEBUG [regionserver8120-smallCompactions-1407799153313]
compactions.Compactor: Compaction progress: 1441233/2197437 (65.59%),
rate=322.72 kB/sec
{noformat}
Status logging code is only active if DEBUG log level is enabled, and will only
print once per minute per compactor thread.
was (Author: apurtell):
Attached patch produces output like:
{noformat}
2014-08-11 16:20:59,739 DEBUG [regionserver8120-smallCompactions-1407799153313]
compactions.Compactor: Compaction progress: 1441233/2197437 (65.59%),
rate=322.72 kB/sec
{noformat}
Status logging code is only active if DEBUG log level is enabled, and will only
print once per minute per compactor thread.
> Better introspection of long running compactions
>
>
> Key: HBASE-11702
> URL: https://issues.apache.org/jira/browse/HBASE-11702
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11702.patch
>
>
> For better introspection of long running compactions, periodically print
> compaction progress for a file at DEBUG level (thread name, file path, total
> compacted KVs, total compacted bytes, completion percent, rate) and expose
> regionserver wide compaction metrics (KVs compacted and bytes compacted) as
> Hadoop value metrics.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[
https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11702:
---
Attachment: HBASE-11702.patch
Attached patch produces output like:
{noformat}
2014-08-11 16:20:59,739 DEBUG [regionserver8120-smallCompactions-1407799153313]
compactions.Compactor: Compaction progress: 1441233/2197437 (65.59%),
rate=322.72 kB/sec
{noformat}
Status logging code is only active if DEBUG log level is enabled, and will only
print once per minute per compactor thread.
> Better introspection of long running compactions
>
>
> Key: HBASE-11702
> URL: https://issues.apache.org/jira/browse/HBASE-11702
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11702.patch
>
>
> For better introspection of long running compactions, periodically print
> compaction progress for a file at DEBUG level (thread name, file path, total
> compacted KVs, total compacted bytes, completion percent, rate) and expose
> regionserver wide compaction metrics (KVs compacted and bytes compacted) as
> Hadoop value metrics.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11702: --- Status: Patch Available (was: Open) > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate) and expose > regionserver wide compaction metrics (KVs compacted and bytes compacted) as > Hadoop value metrics. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093493#comment-14093493 ] Gomathivinayagam Muthuvinayagam commented on HBASE-11699: - Posted a review in reviewboard https://reviews.apache.org/r/24581 > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Labels: patch (was: ) Status: Patch Available (was: Open) > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Labels: patch > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11567) Write bulk load COMMIT events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093474#comment-14093474 ] Alex Newman commented on HBASE-11567: - Sounds good What do you mean failing the RS if the Wal sync fails? > Write bulk load COMMIT events to WAL > > > Key: HBASE-11567 > URL: https://issues.apache.org/jira/browse/HBASE-11567 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alex Newman > Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch > > > Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and > region open/close (HBASE-11512) , we should persist bulk load events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11567) Write bulk load COMMIT events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-11567: Summary: Write bulk load COMMIT events to WAL (was: Write bulk load events to WAL) > Write bulk load COMMIT events to WAL > > > Key: HBASE-11567 > URL: https://issues.apache.org/jira/browse/HBASE-11567 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alex Newman > Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch > > > Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and > region open/close (HBASE-11512) , we should persist bulk load events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11567) Write bulk load events to WAL
[
https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093444#comment-14093444
]
Enis Soztutar commented on HBASE-11567:
---
I would also say that logging the request for bulk load is not as important as
logging the bulk load commit. I do not think of any use for the bulk load
request wal marker.
Do you mind putting an RB for easier review.
I think we would like to use BULK_LOAD for being consistent:
{code}
+ static final byte [] BULK = Bytes.toBytes("HBASE::BULK");
{code}
About jmock it should be ok, but we may want to the escalate the discussion to
dev@ so that other devs are aware.
> Write bulk load events to WAL
> -
>
> Key: HBASE-11567
> URL: https://issues.apache.org/jira/browse/HBASE-11567
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Alex Newman
> Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch
>
>
> Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and
> region open/close (HBASE-11512) , we should persist bulk load events to WAL.
> This is especially important for secondary region replicas, since we can use
> this information to pick up primary regions' files from secondary replicas.
> A design doc for secondary replica replication can be found at HBASE-11183.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11567) Write bulk load events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093425#comment-14093425 ] Enis Soztutar commented on HBASE-11567: --- bq. One question I have is, currently I am logging the request by the user for HBase HFile bulk loading. That being said, does it make sense to have a different WAL entry when it completes successfully? Indeed. The whole point of this issue is to add a WAL marker for the COMMIT of files for bulk load. Since we cannot do an atomic commit of bulk load files in the FS and also in WAL, we can do the commit in FS first, then write the WAL entry next, failing the RS if the WAL sync fails. > Write bulk load events to WAL > - > > Key: HBASE-11567 > URL: https://issues.apache.org/jira/browse/HBASE-11567 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alex Newman > Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch > > > Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and > region open/close (HBASE-11512) , we should persist bulk load events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11679) Replace "HTable" with "HTableInterface" where backwards-compatible
[
https://issues.apache.org/jira/browse/HBASE-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093342#comment-14093342
]
Carter commented on HBASE-11679:
Works for me. I didn't know how far away the Table interface was from being
added. Once that's in, I can merge down again, and the replace the
_HTableInterface_ deltas in my patch with _Table_ and make that my patch
instead.
> Replace "HTable" with "HTableInterface" where backwards-compatible
> --
>
> Key: HBASE-11679
> URL: https://issues.apache.org/jira/browse/HBASE-11679
> Project: HBase
> Issue Type: Improvement
>Reporter: Carter
>Assignee: Carter
> Attachments: HBASE_11679.patch, HBASE_11679.patch,
> HBASE_11679_v3.patch
>
>
> This is a refactor to move more of the code towards using interfaces for
> proper encapsulation of logic.
> The amount of code touched is large, but it should be fairly easy to review.
> It changes variable declarations from HTable to HTableInterface where the
> following holds:
> # The declaration being updated won't break assignment
> # The declaration change does not break the compile (eg trying to access
> non-interface methods)
> The two main situations are to change something like this:
> {code}
> HTable h = new HTable(c, tn);
> {code}
> to
> {code}
> HTableInterface h = new HTable(c, tn);
> {code}
> and this:
> {code}
> public void doSomething(HTable h) { ... }
> {code}
> to this:
> {code}
> public void doSomething(HTableInterface h) { ... }
> {code}
> This gets most of the obvious cases out of the way and prepares for more
> complicated interface refactors in the future. In method signatures, I
> changed parameters, but did _not_ change any public or protected method
> return values, since that would violate criteria #1 above and break
> compatibility.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11512) Write region open/close events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093325#comment-14093325 ] Jeffrey Zhong commented on HBASE-11512: --- I've reviewed the patch. Looks good to me(+1) except a minor thing I posted in RB. Thanks. > Write region open/close events to WAL > - > > Key: HBASE-11512 > URL: https://issues.apache.org/jira/browse/HBASE-11512 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-11512_v1.patch, hbase-11512_v2.patch > > > Similar to writing flush events to WAL (HBASE-11511) and compaction events to > WAL (HBASE-2231), we should write region open and close events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > However, we may need this for regular inter cluster replication as well, see > issues HBASE-10343 and HBASE-9465. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11564) Improve cancellation management in the rpc layer
[ https://issues.apache.org/jira/browse/HBASE-11564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093296#comment-14093296 ] Enis Soztutar commented on HBASE-11564: --- If I remember correctly, I was able to repro this while running IntegrationTestTimeBoundedRequestsWithRegionReplicas with CM. We can come back to this if we see it again in testing. > Improve cancellation management in the rpc layer > > > Key: HBASE-11564 > URL: https://issues.apache.org/jira/browse/HBASE-11564 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.0.0, 2.0.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.99.0, 2.0.0 > > Attachments: 11564.v1.patch, 11564.v2.patch > > > The current client code depends on interrupting the thread for canceling a > request. It's actually possible to rely on a callback in protobuf. > The patch includes as well various performance improvements in replica > management. > On a version before HBASE-11492 the perf was ~35% better. I will redo the > test with the last version. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11679) Replace "HTable" with "HTableInterface" where backwards-compatible
[
https://issues.apache.org/jira/browse/HBASE-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093293#comment-14093293
]
Enis Soztutar commented on HBASE-11679:
---
Sorry, back from vacation. I'll take a look.
Personally, I would prefer we do HBASE-10602 first then do this patch, but use
the new Table interface (which is a much better name than HTableInterface)
{code}
interface Table {
// get, put related stuff
}
@Deprecated
interface HTableInterface extends Table {
// users are encouraged to use the new Table interface
}
class HTable extends Table {
// all HTable constructors are deprecated
// Users are not encouraged to see this class
}
{code}
HBASE-10602 is a bit more involved than just HTable changes (because
HBaseAdmin, and HCM changes as well). I can break those out and do a quick
patch as a subtask if you are ok.
> Replace "HTable" with "HTableInterface" where backwards-compatible
> --
>
> Key: HBASE-11679
> URL: https://issues.apache.org/jira/browse/HBASE-11679
> Project: HBase
> Issue Type: Improvement
>Reporter: Carter
>Assignee: Carter
> Attachments: HBASE_11679.patch, HBASE_11679.patch,
> HBASE_11679_v3.patch
>
>
> This is a refactor to move more of the code towards using interfaces for
> proper encapsulation of logic.
> The amount of code touched is large, but it should be fairly easy to review.
> It changes variable declarations from HTable to HTableInterface where the
> following holds:
> # The declaration being updated won't break assignment
> # The declaration change does not break the compile (eg trying to access
> non-interface methods)
> The two main situations are to change something like this:
> {code}
> HTable h = new HTable(c, tn);
> {code}
> to
> {code}
> HTableInterface h = new HTable(c, tn);
> {code}
> and this:
> {code}
> public void doSomething(HTable h) { ... }
> {code}
> to this:
> {code}
> public void doSomething(HTableInterface h) { ... }
> {code}
> This gets most of the obvious cases out of the way and prepares for more
> complicated interface refactors in the future. In method signatures, I
> changed parameters, but did _not_ change any public or protected method
> return values, since that would violate criteria #1 above and break
> compatibility.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11713) Adding hbase shell unit test coverage for visibility labels.
[
https://issues.apache.org/jira/browse/HBASE-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093280#comment-14093280
]
Hadoop QA commented on HBASE-11713:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661042/HBASE-11713_v2.patch
against trunk revision .
ATTACHMENT ID: 12661042
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10385//console
This message is automatically generated.
> Adding hbase shell unit test coverage for visibility labels.
>
>
> Key: HBASE-11713
> URL: https://issues.apache.org/jira/browse/HBASE-11713
> Project: HBase
> Issue Type: Test
> Components: security, shell
>Reporter: Srikanth Srungarapu
>Assignee: Srikanth Srungarapu
>Priority: Minor
> Attachments: HBASE-11713.patch, HBASE-11713_v2.patch
>
>
> Adding test coverage for visibility labels to hbase shell. Also, refactoring
> existing tests so that all the unit tests related to visibility can be found
> in one place.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11553) Abstract visibility label related services into an interface
[ https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093268#comment-14093268 ] Andrew Purtell commented on HBASE-11553: Let's hold off on commit for more feedback > Abstract visibility label related services into an interface > > > Key: HBASE-11553 > URL: https://issues.apache.org/jira/browse/HBASE-11553 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11553.patch, HBASE-11553.patch, > HBASE-11553_V2.patch, HBASE-11553_V3.patch > > > - storage and retrieval of label dictionary and authentication sets > - marshalling and unmarshalling of visibility expression representations > in operation attributes and cell tags > - management of assignment of authorizations to principals > This will allow us to introduce additional serde implementations for > visibility expressions, for example storing as strings in some places and > compressed/tokenized representation in others in order to support additional > use cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11713) Adding hbase shell unit test coverage for visibility labels.
[ https://issues.apache.org/jira/browse/HBASE-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-11713: Component/s: shell security > Adding hbase shell unit test coverage for visibility labels. > > > Key: HBASE-11713 > URL: https://issues.apache.org/jira/browse/HBASE-11713 > Project: HBase > Issue Type: Test > Components: security, shell >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Attachments: HBASE-11713.patch, HBASE-11713_v2.patch > > > Adding test coverage for visibility labels to hbase shell. Also, refactoring > existing tests so that all the unit tests related to visibility can be found > in one place. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11553) Abstract visibility label related services into an interface
[ https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093252#comment-14093252 ] Sean Busbey commented on HBASE-11553: - I have some feedback in a review I'm ||<-- this close to submitting, just as soon as I can test my intuition on one of the issues. > Abstract visibility label related services into an interface > > > Key: HBASE-11553 > URL: https://issues.apache.org/jira/browse/HBASE-11553 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11553.patch, HBASE-11553.patch, > HBASE-11553_V2.patch, HBASE-11553_V3.patch > > > - storage and retrieval of label dictionary and authentication sets > - marshalling and unmarshalling of visibility expression representations > in operation attributes and cell tags > - management of assignment of authorizations to principals > This will allow us to introduce additional serde implementations for > visibility expressions, for example storing as strings in some places and > compressed/tokenized representation in others in order to support additional > use cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11713) Adding hbase shell unit test coverage for visibility labels.
[ https://issues.apache.org/jira/browse/HBASE-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-11713: Attachment: HBASE-11713_v2.patch Attaching the new patch as per suggestions. > Adding hbase shell unit test coverage for visibility labels. > > > Key: HBASE-11713 > URL: https://issues.apache.org/jira/browse/HBASE-11713 > Project: HBase > Issue Type: Test >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Attachments: HBASE-11713.patch, HBASE-11713_v2.patch > > > Adding test coverage for visibility labels to hbase shell. Also, refactoring > existing tests so that all the unit tests related to visibility can be found > in one place. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11719) Remove some unused paths in AsyncClient
[ https://issues.apache.org/jira/browse/HBASE-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093118#comment-14093118 ] Sergey Shelukhin commented on HBASE-11719: -- +1 > Remove some unused paths in AsyncClient > --- > > Key: HBASE-11719 > URL: https://issues.apache.org/jira/browse/HBASE-11719 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.0.0, 2.0.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Trivial > Fix For: 1.0.0, 2.0.0 > > Attachments: simplifyMultiReplica.patch > > > [~sershe] you're ok with these changes? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11712) Backport HBASE-3585 to 0.89-fb branch
[ https://issues.apache.org/jira/browse/HBASE-11712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11712: Summary: Backport HBASE-3585 to 0.89-fb branch (was: Code possibly occur out of range exception) updated the ticket title to reflect backport need. > Backport HBASE-3585 to 0.89-fb branch > - > > Key: HBASE-11712 > URL: https://issues.apache.org/jira/browse/HBASE-11712 > Project: HBase > Issue Type: Bug >Affects Versions: 0.89-fb >Reporter: Hyeonjong >Priority: Minor > Labels: beginner > Fix For: 0.89-fb > > > isLegalFamilyName might have a zero length string as the argument, but > current code does not check its length and directly try to access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11714) RpcRetryingCaller#callWithoutRetries set rpc timeout to 2 seconds incorrectly
[
https://issues.apache.org/jira/browse/HBASE-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11714:
---
Fix Version/s: (was: 0.98.4)
> RpcRetryingCaller#callWithoutRetries set rpc timeout to 2 seconds incorrectly
> -
>
> Key: HBASE-11714
> URL: https://issues.apache.org/jira/browse/HBASE-11714
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Affects Versions: 0.98.3
>Reporter: Qiang Tian
>Assignee: Qiang Tian
> Attachments: hbase-11714-0.98.patch
>
>
> Discussed on the user@hbase mailing list
> (http://markmail.org/thread/w3cqjxwo2smkn2jw)
> {quote}
> "Recently switched from 0.94 and 0.98, and finding that periodically things
> are having issues - lots of retry exceptions" :
> {quote}
> client log:
> {quote}
> 2014-08-08 17:22:43 o.a.h.h.c.AsyncProcess [INFO] #105158,
> table=rt_global_monthly_campaign_deliveries, attempt=10/35 failed 500 ops,
> last exception: java.net.SocketTimeoutException: Call to
> ip-10-201-128-23.us-west-1.compute.internal/10.201.128.23:60020 failed
> because java.net.SocketTimeoutException: 2000 millis timeout while waiting
> for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/10.248.130.152:46014
> remote=ip-10-201-128-23.us-west-1.compute.internal/10.201.128.23:60020] on
> ip-10-201-128-23.us-west-1.compute.internal,60020,1405642103651, tracking
> started Fri Aug 08 17:21:55 UTC 2014, retrying after 10043 ms, replay 500
> ops.
> {quote}
> analysis:
> there are 2 methods in RpcRetryingCaller: callWithRetries and
> callWithoutRetries.
> it looks the timeout setup of callWithRetries is good, while
> callWithoutRetries is wrong(multi RPC for this user): caller cannot specify a
> valid timeout, but callWithoutRetries still calls beforeCall, which looks a
> method for callWithRetries only, to set timeout. since
> RpcRetryingCaller#callTimeout is not set, thread local timeout is set to
> 2s(MIN_RPC_TIMEOUT) via RpcClient.setRpcTimeout, which is the final
> pinginterval set to the socket.
> when there are heavy write workload and the rpc cannot complete in 2s, the
> client close the connection, so the server side connection is reset and
> finally exposes the problem in HBASE-11705
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11712) Code possibly occur out of range exception
[ https://issues.apache.org/jira/browse/HBASE-11712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093081#comment-14093081 ] Hyeonjong commented on HBASE-11712: --- [~busbey] It was for 0.89-fb branch. Sorry for skipping the version :) > Code possibly occur out of range exception > -- > > Key: HBASE-11712 > URL: https://issues.apache.org/jira/browse/HBASE-11712 > Project: HBase > Issue Type: Bug >Affects Versions: 0.89-fb >Reporter: Hyeonjong >Priority: Minor > Labels: beginner > Fix For: 0.89-fb > > > isLegalFamilyName might have a zero length string as the argument, but > current code does not check its length and directly try to access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11712) Code possibly occur out of range exception
[ https://issues.apache.org/jira/browse/HBASE-11712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyeonjong updated HBASE-11712: -- Fix Version/s: 0.89-fb > Code possibly occur out of range exception > -- > > Key: HBASE-11712 > URL: https://issues.apache.org/jira/browse/HBASE-11712 > Project: HBase > Issue Type: Bug >Affects Versions: 0.89-fb >Reporter: Hyeonjong >Priority: Minor > Labels: beginner > Fix For: 0.89-fb > > > isLegalFamilyName might have a zero length string as the argument, but > current code does not check its length and directly try to access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11712) Code possibly occur out of range exception
[ https://issues.apache.org/jira/browse/HBASE-11712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyeonjong updated HBASE-11712: -- Affects Version/s: 0.89-fb > Code possibly occur out of range exception > -- > > Key: HBASE-11712 > URL: https://issues.apache.org/jira/browse/HBASE-11712 > Project: HBase > Issue Type: Bug >Affects Versions: 0.89-fb >Reporter: Hyeonjong >Priority: Minor > Labels: beginner > Fix For: 0.89-fb > > > isLegalFamilyName might have a zero length string as the argument, but > current code does not check its length and directly try to access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11626) Region will be closed if user try to move it to an error destination server
[
https://issues.apache.org/jira/browse/HBASE-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093070#comment-14093070
]
Andrew Purtell commented on HBASE-11626:
Sorry for the misunderstanding [~cuijianwei], I agree with your opinion.
> Region will be closed if user try to move it to an error destination server
> ---
>
> Key: HBASE-11626
> URL: https://issues.apache.org/jira/browse/HBASE-11626
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 0.98.4
>Reporter: cuijianwei
>Priority: Minor
>
> When moving region to a destination server by hbase shell command, such as:
> {code}
> move '7dbe79eb3a77df085c8546b2ee540e7f', 'localhost,58220,1406806842691'
> {code}
> user could pass an error destination server name, such as misspellinging the
> server name. Then, HMaster will close the region firstly and then assign the
> region to a random server when finding the destination server is not online,
> and the region can not serve before it is online again. Do we need to check
> the destination server name corresponds to an online server before closing
> region?(in HMaster.move(...))
> {code}
> ...
> if (destServerName == null || destServerName.length == 0) {
>
> } else {
> dest = ServerName.valueOf(Bytes.toString(destServerName));
> if (dest.equals(regionState.getServerName())) {
> ...
> }
> // ### check the destination sever is online before closing region ###
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-11716) LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
[
https://issues.apache.org/jira/browse/HBASE-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-11716.
Resolution: Fixed
Fix Version/s: 2.0.0
0.99.0
Hadoop Flags: Reviewed
Committed to 0.98+ . Thanks for the patch Anoop!
> LoadTestDataGeneratorWithVisibilityLabels should handle Delete mutations
>
>
> Key: HBASE-11716
> URL: https://issues.apache.org/jira/browse/HBASE-11716
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.5
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Labels: IntegrationTest
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11716.patch
>
>
> {code}
> public Mutation beforeMutate(long rowkeyBase, Mutation m) throws
> IOException {
> if (!(m instanceof Delete)) {
> m.setCellVisibility(new CellVisibility(this.visibilityExps[(int)
> rowkeyBase
> % this.visibilityExps.length]));
> }
> return m;
> }
> {code}
> Now we have delete CellVisibility matching and so we should add
> CellVisibility for Mutations also. Just removing the if check part will solve
> the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: HBASE_11699.patch > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: (was: HBASE_11699.patch) > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: (was: HBASE_11699.patch) > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: HBASE_11699.patch > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093034#comment-14093034 ] Andrew Purtell commented on HBASE-11438: HBASE-10883 doesn't apply if we are allowing any UTF-8 string as an authorization. There is no invalid characters to protect against, any character in an authorization will be valid, if such an authorization/label has been defined. Only when parsing visibility expressions will we need to differentiate between the characters that make up a label identifier and operators in the expression. There we require operator-like chars in the identifier to be quoted. > [Visibility Controller] Support UTF8 character as Visibility Labels > --- > > Key: HBASE-11438 > URL: https://issues.apache.org/jira/browse/HBASE-11438 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.4 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.6 > > Attachments: HBASE-11438_v1.patch, HBASE-11438_v2.patch, > HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, > HBASE-11438_v6.patch, HBASE-11438_v7.patch, HBASE-11438_v8.patch, > HBASE-11438_v9.patch > > > This would be an action item that we would be addressing so that the > visibility labels could have UTF8 characters in them. Also allow the user to > use a client supplied API that allows to specify the visibility labels inside > double quotes such that UTF8 characters and cases like &, |, ! and double > quotes itself could be specified with proper escape sequence. Accumulo too > provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093030#comment-14093030 ] Gomathivinayagam Muthuvinayagam commented on HBASE-11699: - I attached the patch, could you review them? Thanks. > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11699) Region servers exclusion list to HMaster.
[ https://issues.apache.org/jira/browse/HBASE-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gomathivinayagam Muthuvinayagam updated HBASE-11699: Attachment: HBASE_11699.patch > Region servers exclusion list to HMaster. > - > > Key: HBASE-11699 > URL: https://issues.apache.org/jira/browse/HBASE-11699 > Project: HBase > Issue Type: New Feature > Components: Admin, Client, regionserver, Zookeeper >Affects Versions: 0.98.3 >Reporter: Gomathivinayagam Muthuvinayagam >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE_11699.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently HBase does not support adding set of region servers to be in the > exclusion list. So that administrators can prevent accidental startups of > some region servers to join the cluster. There was initially some work done, > and it is available in https://issues.apache.org/jira/browse/HBASE-3833. It > was not done after that. > I am planning to contribute it as a patch, and I would like to do some > improvements as well. Instead of storing the exclusion entries on a file, I > am planning to store it on zookeeper. Can anyone suggest thoughts on this? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11072) Abstract WAL splitting from ZK
[ https://issues.apache.org/jira/browse/HBASE-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Soldatov updated HBASE-11072: Status: Patch Available (was: Open) Yet another attempt to rebuild > Abstract WAL splitting from ZK > -- > > Key: HBASE-11072 > URL: https://issues.apache.org/jira/browse/HBASE-11072 > Project: HBase > Issue Type: Sub-task > Components: Consensus, Zookeeper >Affects Versions: 0.99.0 >Reporter: Mikhail Antonov >Assignee: Sergey Soldatov > Attachments: HBASE-11072-1_v2.patch, HBASE-11072-1_v3.patch, > HBASE-11072-1_v4.patch, HBASE-11072-2_v2.patch, HBASE-11072-v1.patch, > HBASE-11072-v2.patch, HBASE-11072-v3.patch, HBASE-11072-v4.patch, > HBASE-11072-v5.patch, HBASE-11072-v6.patch, HBASE_11072-1.patch > > > HM side: > - SplitLogManager > RS side: > - SplitLogWorker > - HLogSplitter and a few handler classes. > This jira may need to be split further apart into smaller ones. -- This message was sent by Atlassian JIRA (v6.2#6252)
