[jira] [Commented] (HBASE-1403) Move transactional and indexer hbase out of core into contrib or out to their own project
[ https://issues.apache.org/jira/browse/HBASE-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087447#comment-14087447 ] Liang Li commented on HBASE-1403: - Hi,stack, will you please tell me the detail reason why you move the index down? Also, the current version is 0.98.1, Do you consider to recover the index part to main version? > Move transactional and indexer hbase out of core into contrib or out to their > own project > - > > Key: HBASE-1403 > URL: https://issues.apache.org/jira/browse/HBASE-1403 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 0.20.0 > > > Its a bit of work bringing along these hbase subclasses as changes happen in > core. They are substantial enough contributions, they could be their own > projects -- if there was the will to keep them up. Otherwise, we could move > them down into contrib. -- that'd better demark core and this hbase > customizations. > I wrote Clint asking what he thought. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-11683) Metrics for MOB
[ https://issues.apache.org/jira/browse/HBASE-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du reassigned HBASE-11683: Assignee: Jingcheng Du > Metrics for MOB > --- > > Key: HBASE-11683 > URL: https://issues.apache.org/jira/browse/HBASE-11683 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jingcheng Du > > We need to make sure to capture metrics about mobs. > Some basic ones include: > # of mob writes > # of mob reads > # avg size of mob (?) > # mob files > # of mob compactions / sweeps -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11683) Metrics for MOB
[ https://issues.apache.org/jira/browse/HBASE-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087492#comment-14087492 ] Jingcheng Du commented on HBASE-11683: -- Currently we have defined the metrics on the mob flusher. 1. how many times of flushing occurs. 2. how many cells are flushed to the mob file. 3. how many size of mob cells are flushed to the mob file. about #5 The compaction/sweeps is a tool out of hbase and uses the mapreduce. We use the context counter to do the metrics work. I'm thinking how to implement the #2 mob reads, is it okay to record how many times the scanner read from the mob files? I don't see HBase has metrics in the normal scanner, is it necessary for the mob read? Please advise. Thanks. > Metrics for MOB > --- > > Key: HBASE-11683 > URL: https://issues.apache.org/jira/browse/HBASE-11683 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jingcheng Du > > We need to make sure to capture metrics about mobs. > Some basic ones include: > # of mob writes > # of mob reads > # avg size of mob (?) > # mob files > # of mob compactions / sweeps -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
Liu Shaohui created HBASE-11685:
---
Summary: Incr/decr on the reference count of
HConnectionImplementation need be atomic
Key: HBASE-11685
URL: https://issues.apache.org/jira/browse/HBASE-11685
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
Currently, the incr/decr operation on the ref count of
HConnectionImplementation are not atomic. This may cause that the ref count
always be larger than 0 and the connection never be closed.
{code}
/**
* Increment this client's reference count.
*/
void incCount() {
++refCount;
}
/**
* Decrement this client's reference count.
*/
void decCount() {
if (refCount > 0) {
--refCount;
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087558#comment-14087558 ] ramkrishna.s.vasudevan commented on HBASE-11438: bq.But we should validate (like we did in past) Then the validation should check for double quotes and allow that. But I don't feel that would be needed. Because then we should actually parse the string rewrite the string without quotes. Same applies with authorizations also. Other argument I would suggest is that if we are allowing everything in the addLabels and authorizations what is there to be validated? And the point to see here is that in the addlabels and authorizations there is no expression that we are trying to build. So for using &,!,| should be made simple IMO. > [Visibility Controller] Support UTF8 character as Visibility Labels > --- > > Key: HBASE-11438 > URL: https://issues.apache.org/jira/browse/HBASE-11438 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.4 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.6 > > Attachments: HBASE-11438_v1.patch, HBASE-11438_v2.patch, > HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, > HBASE-11438_v6.patch, HBASE-11438_v7.patch > > > This would be an action item that we would be addressing so that the > visibility labels could have UTF8 characters in them. Also allow the user to > use a client supplied API that allows to specify the visibility labels inside > double quotes such that UTF8 characters and cases like &, |, ! and double > quotes itself could be specified with proper escape sequence. Accumulo too > provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated HBASE-11685:
Attachment: HBASE-11685-trunk-v1.diff
Change the refCount from int to AtomicInteger
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated HBASE-11685:
Fix Version/s: 2.0.0
Status: Patch Available (was: Open)
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087576#comment-14087576 ] Anoop Sam John commented on HBASE-11438: So you plan to undo HBASE-10883 fix? > [Visibility Controller] Support UTF8 character as Visibility Labels > --- > > Key: HBASE-11438 > URL: https://issues.apache.org/jira/browse/HBASE-11438 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.4 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.6 > > Attachments: HBASE-11438_v1.patch, HBASE-11438_v2.patch, > HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, > HBASE-11438_v6.patch, HBASE-11438_v7.patch > > > This would be an action item that we would be addressing so that the > visibility labels could have UTF8 characters in them. Also allow the user to > use a client supplied API that allows to specify the visibility labels inside > double quotes such that UTF8 characters and cases like &, |, ! and double > quotes itself could be specified with proper escape sequence. Accumulo too > provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087603#comment-14087603
]
Hadoop QA commented on HBASE-11685:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660120/HBASE-11685-trunk-v1.diff
against trunk revision .
ATTACHMENT ID: 12660120
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestClientNoCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10313//console
This message is automatically generated.
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated HBASE-11685:
Attachment: HBASE-11685-trunk-v2.diff
Fix failed test: TestClientNoCluster
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff, HBASE-11685-trunk-v2.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-9416) Use the disruptor library in the rcpServer
[ https://issues.apache.org/jira/browse/HBASE-9416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon resolved HBASE-9416. Resolution: Won't Fix No work on this for a while, the patch need to be redone totally with the recent changes on this part => solving as won't fix. > Use the disruptor library in the rcpServer > -- > > Key: HBASE-9416 > URL: https://issues.apache.org/jira/browse/HBASE-9416 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.98.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Attachments: disrupt.patch > > > Nasty & hacky patch on top of the 0.96 to get some feedback on adding this > third party. > I ran a test doing "gets" on an empty region. > With the current implementation, we're spending time in the > LinkedBlockingQueue#put. I was able to do 150K operations per second. > Using the disruptor allowed me to go to 190 ops/s, i.e. a little be more than > a 25% improvement. > Likely there are other improvements in this class as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carter updated HBASE-11657:
---
Status: Open (was: Patch Available)
> Put HTable region methods in an interface
> -
>
> Key: HBASE-11657
> URL: https://issues.apache.org/jira/browse/HBASE-11657
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Carter
>Assignee: Carter
> Fix For: 0.99.0
>
> Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
> HBASE_11657_v3.patch
>
>
> Most of the HTable methods are now abstracted by HTableInterface, with the
> notable exception of the following methods that pertain to region metadata:
> {code}
> HRegionLocation getRegionLocation(final String row)
> HRegionLocation getRegionLocation(final byte [] row)
> HRegionLocation getRegionLocation(final byte [] row, boolean reload)
> byte [][] getStartKeys()
> byte[][] getEndKeys()
> Pair getStartEndKeys()
> void clearRegionCache()
> {code}
> and a default scope method which maybe should be bundled with the others:
> {code}
> List listRegionLocations()
> {code}
> Since the consensus seems to be that these would muddy HTableInterface with
> non-core functionality, where should it go? MapReduce looks up the region
> boundaries, so it needs to be exposed somewhere.
> Let me throw out a straw man to start the conversation. I propose:
> {code}
> org.apache.hadoop.hbase.client.HRegionInterface
> {code}
> Have HTable implement this interface. Also add these methods to HConnection:
> {code}
> HRegionInterface getTableRegion(TableName tableName)
> HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
> {code}
> [~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carter updated HBASE-11657:
---
Attachment: HBASE_11657_v3.patch
> Put HTable region methods in an interface
> -
>
> Key: HBASE-11657
> URL: https://issues.apache.org/jira/browse/HBASE-11657
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Carter
>Assignee: Carter
> Fix For: 0.99.0
>
> Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
> HBASE_11657_v3.patch
>
>
> Most of the HTable methods are now abstracted by HTableInterface, with the
> notable exception of the following methods that pertain to region metadata:
> {code}
> HRegionLocation getRegionLocation(final String row)
> HRegionLocation getRegionLocation(final byte [] row)
> HRegionLocation getRegionLocation(final byte [] row, boolean reload)
> byte [][] getStartKeys()
> byte[][] getEndKeys()
> Pair getStartEndKeys()
> void clearRegionCache()
> {code}
> and a default scope method which maybe should be bundled with the others:
> {code}
> List listRegionLocations()
> {code}
> Since the consensus seems to be that these would muddy HTableInterface with
> non-core functionality, where should it go? MapReduce looks up the region
> boundaries, so it needs to be exposed somewhere.
> Let me throw out a straw man to start the conversation. I propose:
> {code}
> org.apache.hadoop.hbase.client.HRegionInterface
> {code}
> Have HTable implement this interface. Also add these methods to HConnection:
> {code}
> HRegionInterface getTableRegion(TableName tableName)
> HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
> {code}
> [~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carter updated HBASE-11657:
---
Status: Patch Available (was: Open)
Good point. Stability changed to "Evolving". Patch v3 submitted.
> Put HTable region methods in an interface
> -
>
> Key: HBASE-11657
> URL: https://issues.apache.org/jira/browse/HBASE-11657
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Carter
>Assignee: Carter
> Fix For: 0.99.0
>
> Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
> HBASE_11657_v3.patch
>
>
> Most of the HTable methods are now abstracted by HTableInterface, with the
> notable exception of the following methods that pertain to region metadata:
> {code}
> HRegionLocation getRegionLocation(final String row)
> HRegionLocation getRegionLocation(final byte [] row)
> HRegionLocation getRegionLocation(final byte [] row, boolean reload)
> byte [][] getStartKeys()
> byte[][] getEndKeys()
> Pair getStartEndKeys()
> void clearRegionCache()
> {code}
> and a default scope method which maybe should be bundled with the others:
> {code}
> List listRegionLocations()
> {code}
> Since the consensus seems to be that these would muddy HTableInterface with
> non-core functionality, where should it go? MapReduce looks up the region
> boundaries, so it needs to be exposed somewhere.
> Let me throw out a straw man to start the conversation. I propose:
> {code}
> org.apache.hadoop.hbase.client.HRegionInterface
> {code}
> Have HTable implement this interface. Also add these methods to HConnection:
> {code}
> HRegionInterface getTableRegion(TableName tableName)
> HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
> {code}
> [~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11611: Attachment: hbase-11611.patch > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11606) Enable ZK-less region assignment by default
[ https://issues.apache.org/jira/browse/HBASE-11606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087733#comment-14087733 ] Jimmy Xiang commented on HBASE-11606: - I see. I attached a patch for HBASE-11611, with which I have verified that our internal jenkins is green. We should keep jenkins ok. > Enable ZK-less region assignment by default > --- > > Key: HBASE-11606 > URL: https://issues.apache.org/jira/browse/HBASE-11606 > Project: HBase > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0 > > Attachments: hbase-11606.patch > > > Let's enable ZK-less region assignment by default in the master branch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[
https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087739#comment-14087739
]
Hadoop QA commented on HBASE-11611:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660145/hbase-11611.patch
against trunk revision .
ATTACHMENT ID: 12660145
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 97 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10316//console
This message is automatically generated.
> Clean up ZK-based region assignment
> ---
>
> Key: HBASE-11611
> URL: https://issues.apache.org/jira/browse/HBASE-11611
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 2.0.0
>
> Attachments: hbase-11611.patch
>
>
> We can clean up the ZK-based region assignment code and use the ZK-less one
> in the master branch, to make the code easier to understand and maintain.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087747#comment-14087747 ] Jimmy Xiang commented on HBASE-11611: - Patch v1 is on RB: https://reviews.apache.org/r/24382/ > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11682) Explain hotspotting
[ https://issues.apache.org/jira/browse/HBASE-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087751#comment-14087751 ] Jonathan Hsieh commented on HBASE-11682: Nice addition. Personally, I don't really like the sematext definition of salting it conflates salting [1] with hashing[2] which are two separate things. *salting* adds random data to the start of a rowkey. this means depending on the 'salt factor' you could end up writing to n different row keys (and ideally n different regions). When reading you would generally want to read all n rows and coalesce the values. This is helpful if you have individual hot keys. It is often a bad smell because it is a trick used to try to mitigate having the date as the first part of a row key, but does have valid use cases. (ex: rowkey is name and you have a handful of individual celebreties - obama, bieber, gaga - that need to have their load spread). This preserves ordering but multiplies the number of reads required wrt # of writes. *hashing* applies a random one way function to the rowkey such that a particular row will get the same 'random' value prepended. The original row would get mapped to a single row. This is good for when you have clusters of related keys that in aggregate form a hotspot. (Example: rowkey is name and you have way to many joe's, john's, jon's, jonah's, jonathan's, and jonathons's all on the same region -- using a hash would spread all the j names around). this throws out the ability to effectively take advantage of the row ordering properties. Another trick is to take numeric or fixed length values and make the least significant digit (e.g. the one that changes the most) in least significant digit order (little endian). This effectively randomizes row key names but also sacrifices row ordering properties. [1] http://en.wikipedia.org/wiki/Salt_(cryptography) [2] http://en.wikipedia.org/wiki/Hash_function > Explain hotspotting > --- > > Key: HBASE-11682 > URL: https://issues.apache.org/jira/browse/HBASE-11682 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11682-1.patch, HBASE-11682.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11611: Attachment: hbase-11611_v1.patch Attached the same patch generated with --no-prefix. Our patch script is changed again? > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[
https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087757#comment-14087757
]
Hadoop QA commented on HBASE-11611:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660148/hbase-11611_v1.patch
against trunk revision .
ATTACHMENT ID: 12660148
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 97 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10317//console
This message is automatically generated.
> Clean up ZK-based region assignment
> ---
>
> Key: HBASE-11611
> URL: https://issues.apache.org/jira/browse/HBASE-11611
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 2.0.0
>
> Attachments: hbase-11611.patch, hbase-11611_v1.patch
>
>
> We can clean up the ZK-based region assignment code and use the ZK-less one
> in the master branch, to make the code easier to understand and maintain.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11611: Status: Patch Available (was: Open) Attached a patch that removed ZK related assignment code, fixed tests. > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087787#comment-14087787 ] Jimmy Xiang commented on HBASE-11611: - That's weird. The patch applies locally. > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087792#comment-14087792
]
Hadoop QA commented on HBASE-11657:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660139/HBASE_11657_v3.patch
against trunk revision .
ATTACHMENT ID: 12660139
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.client.TestReplicasClient
org.apache.hadoop.hbase.regionserver.TestRegionReplicas
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10315//console
This message is automatically generated.
> Put HTable region methods in an interface
> -
>
> Key: HBASE-11657
> URL: https://issues.apache.org/jira/browse/HBASE-11657
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Carter
>Assignee: Carter
> Fix For: 0.99.0
>
> Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
> HBASE_11657_v3.patch
>
>
> Most of the HTable methods are now abstracted by HTableInterface, with the
> notable exception of the following methods that pertain to region metadata:
> {code}
> HRegionLocation getRegionLocation(final String row)
> HRegionLocation getRegionLocation(final byte [] row)
> HRegionLocation getRegionLocation(final byte [] row, boolean reload)
> byte [][] getStartKeys()
> byte[][] getEndKeys()
> Pair getStartEndKeys()
> void clearRegionCache()
> {code}
> and a default scope method which maybe should be bundled with the others:
> {code}
> List listRegionLocations()
> {code}
> Since the consensus seems to be that these would muddy HTableInterface with
> non-core functionality, where should it go? MapReduce looks up the region
> boundaries, so it needs to be exposed somewhere.
> Let me throw out a straw man to start the conversation. I propose:
> {code}
> org.apache.hadoop.hbase.client.HRegionInterface
> {code}
> Have HTable implement this interface. Also add these methods to HConnection:
> {code}
> HRegionInterface getTabl
[jira] [Updated] (HBASE-11553) Abstract visibility label related services into an interface
[ https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11553: --- Attachment: HBASE-11553_V2.patch > Abstract visibility label related services into an interface > > > Key: HBASE-11553 > URL: https://issues.apache.org/jira/browse/HBASE-11553 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11553.patch, HBASE-11553.patch, > HBASE-11553_V2.patch > > > - storage and retrieval of label dictionary and authentication sets > - marshalling and unmarshalling of visibility expression representations > in operation attributes and cell tags > - management of assignment of authorizations to principals > This will allow us to introduce additional serde implementations for > visibility expressions, for example storing as strings in some places and > compressed/tokenized representation in others in order to support additional > use cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11553) Abstract visibility label related services into an interface
[ https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11553: --- Status: Patch Available (was: Open) > Abstract visibility label related services into an interface > > > Key: HBASE-11553 > URL: https://issues.apache.org/jira/browse/HBASE-11553 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11553.patch, HBASE-11553.patch, > HBASE-11553_V2.patch > > > - storage and retrieval of label dictionary and authentication sets > - marshalling and unmarshalling of visibility expression representations > in operation attributes and cell tags > - management of assignment of authorizations to principals > This will allow us to introduce additional serde implementations for > visibility expressions, for example storing as strings in some places and > compressed/tokenized representation in others in order to support additional > use cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11553) Abstract visibility label related services into an interface
[
https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087802#comment-14087802
]
Hadoop QA commented on HBASE-11553:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660153/HBASE-11553_V2.patch
against trunk revision .
ATTACHMENT ID: 12660153
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LabelExpander.java:[88,28]
error: cannot find symbol
[ERROR] symbol: variable VIS_SERIALIZATION_TAG
[ERROR] symbol: variable VISIBILITY_TAG_TYPE
[ERROR] symbol: variable VISIBILITY_TAG_TYPE
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LabelExpander.java:[88,28]
error: cannot find symbol
[ERROR] symbol: variable VIS_SERIALIZATION_TAG
[ERROR] location: class VisibilityUtils
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LabelExpander.java:[92,38]
error: cannot find symbol
[ERROR] symbol: variable VISIBILITY_TAG_TYPE
[ERROR] location: class VisibilityUtils
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LabelExpander.java:[100,42]
error: cannot find symbol
[ERROR] symbol: variable VISIBILITY_TAG_TYPE
[ERROR] location: class VisibilityUtils
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LabelExpander.java:[107,40]
error: cannot find symbol
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10318//console
This message is automatically generated.
> Abstract visibility label related services into an interface
>
>
> Key: HBASE-11553
> URL: https://issues.apache.org/jira/browse/HBASE-11553
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11553.patch, HBASE-11553.patch,
> HBASE-11553_V2.patch
>
>
> - storage and retrieval of label dictionary and authentication sets
> - marshalling and unmarshalling of visibility expression representations
> in operation attributes and cell tags
> - management of assignment of authorizations to principals
> This will allow us to introduce additional serde implementations for
> visibility expressions, for example storing as strings in some places and
> compressed/tokenized representation in others in order to support additional
> use cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087831#comment-14087831 ] ramkrishna.s.vasudevan commented on HBASE-11438: bq.So you plan to undo HBASE-10883 fix? Yes but not fully because the validation is still getting used in checking if the visibility expression without quotes has valid characters as specified in the VisibilityLabelsValidator. If you see the description of HBASE-10883 it is mainly to disallow usage of &,!,| type of characters so that users should not use them. So by removing that check we are ensuring that user can use them now. But in authorizations there are no expressions and only string literals. > [Visibility Controller] Support UTF8 character as Visibility Labels > --- > > Key: HBASE-11438 > URL: https://issues.apache.org/jira/browse/HBASE-11438 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.4 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.98.6 > > Attachments: HBASE-11438_v1.patch, HBASE-11438_v2.patch, > HBASE-11438_v3.patch, HBASE-11438_v4.patch, HBASE-11438_v5.patch, > HBASE-11438_v6.patch, HBASE-11438_v7.patch > > > This would be an action item that we would be addressing so that the > visibility labels could have UTF8 characters in them. Also allow the user to > use a client supplied API that allows to specify the visibility labels inside > double quotes such that UTF8 characters and cases like &, |, ! and double > quotes itself could be specified with proper escape sequence. Accumulo too > provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11682) Explain hotspotting
[ https://issues.apache.org/jira/browse/HBASE-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087833#comment-14087833 ] Nick Dimiduk commented on HBASE-11682: -- Please note that "salting" as we use the term in HBase is different from "salting" in the cryptographic sense. Our usage pattern is more accurately described as "bucketing" (I think this is the term Phoenix uses), or "binning" [0]. This horse has been beaten to death on the user and dev mailing lists, so I won't belabor the point. [0]: http://en.wikipedia.org/wiki/Data_binning > Explain hotspotting > --- > > Key: HBASE-11682 > URL: https://issues.apache.org/jira/browse/HBASE-11682 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11682-1.patch, HBASE-11682.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11682) Explain hotspotting
[ https://issues.apache.org/jira/browse/HBASE-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087839#comment-14087839 ] Nick Dimiduk commented on HBASE-11682: -- bq. However, using totally random row keys for data which is accessed sequentially would remove the benefit of HBase's row-sorting algorithm and cause very poor performance, as each get or scan would need to query all regions. Prefixing with random byte prevents any meaningful use of scans; gets become your only option. This approach is indistinguishable from hashing the rowkey. I like the rest of the updates, +1 Thanks a lot [~misty]! > Explain hotspotting > --- > > Key: HBASE-11682 > URL: https://issues.apache.org/jira/browse/HBASE-11682 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11682-1.patch, HBASE-11682.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087844#comment-14087844 ] Mikhail Antonov commented on HBASE-11611: - BTW, the patch doesn't apply with either 'git apply -p0' or 'git apply -p1', can only apply with Intellij Idea. > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087859#comment-14087859 ] Jimmy Xiang commented on HBASE-11611: - Check dev-support/smart-apply-patch.sh. That's the command to apply the patch: $PATCH -p0 -E --dry-run < $PATCH_FILE 2>&1 > $TMP It should be "/usr/bin/patch" instead of "git apply". > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11553) Abstract visibility label related services into an interface
[ https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11553: --- Attachment: HBASE-11553_V2.patch > Abstract visibility label related services into an interface > > > Key: HBASE-11553 > URL: https://issues.apache.org/jira/browse/HBASE-11553 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11553.patch, HBASE-11553.patch, > HBASE-11553_V2.patch > > > - storage and retrieval of label dictionary and authentication sets > - marshalling and unmarshalling of visibility expression representations > in operation attributes and cell tags > - management of assignment of authorizations to principals > This will allow us to introduce additional serde implementations for > visibility expressions, for example storing as strings in some places and > compressed/tokenized representation in others in order to support additional > use cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11553) Abstract visibility label related services into an interface
[ https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-11553: --- Attachment: (was: HBASE-11553_V2.patch) > Abstract visibility label related services into an interface > > > Key: HBASE-11553 > URL: https://issues.apache.org/jira/browse/HBASE-11553 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11553.patch, HBASE-11553.patch, > HBASE-11553_V2.patch > > > - storage and retrieval of label dictionary and authentication sets > - marshalling and unmarshalling of visibility expression representations > in operation attributes and cell tags > - management of assignment of authorizations to principals > This will allow us to introduce additional serde implementations for > visibility expressions, for example storing as strings in some places and > compressed/tokenized representation in others in order to support additional > use cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11678) BucketCache ramCache fills heap after running a few hours
[ https://issues.apache.org/jira/browse/HBASE-11678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11678: -- Attachment: gc_over_12_hours_unevenblocks_with_lots_of_evictions.png Ran for 12 hours. GC plot looks clean. It may be tending up slightly... will run for longer next time but looking at heap dump, this ramCache issue is no longer the culprit. Looking to make a unit test. Its a bit awkward given how deep down doDrain is inside BucketCache but worth doing I'd say. > BucketCache ramCache fills heap after running a few hours > - > > Key: HBASE-11678 > URL: https://issues.apache.org/jira/browse/HBASE-11678 > Project: HBase > Issue Type: Bug > Components: BlockCache >Affects Versions: 0.99.0, 0.98.5, 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: > 0001-When-we-failed-add-an-entry-failing-with-a-CacheFull.patch, > gc_over_12_hours_unevenblocks_with_lots_of_evictions.png > > > Testing BucketCache, my heap filled after running for hours. Dumping heap, > culprit is the ramCache Map in BucketCache. Tried running with more writer > threads but made no difference. Trying to figure now how our accounting is > going wonky. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11682) Explain hotspotting
[ https://issues.apache.org/jira/browse/HBASE-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087891#comment-14087891 ] Jonathan Hsieh commented on HBASE-11682: bq. Please note that "salting" as we use the term in HBase is different from "salting" in the cryptographic sense. Our usage pattern is more accurately described as "bucketing" (I think this is the term Phoenix uses), or "binning" [0]. This horse has been beaten to death on the user and dev mailing lists, so I won't belabor the point. I agree they are different but the idea of a random (as opposed to a deterministic hash) prepended is very similar to the crypto "salting". Instead of using "salting" how about we use the term "striping"? What I've was referring and described takes one logical row and stripes it across many rows so that write throughput can be increased. The penalty is that we loose some consistency and also if we want a "whole" answer read all the rows the logical rows was striped over. I've seen the pattern deployed at several customers. >From what phoenix uses and what I think you mean, "bucketing', and "binning" >are equivalent to prepending a hash. > Explain hotspotting > --- > > Key: HBASE-11682 > URL: https://issues.apache.org/jira/browse/HBASE-11682 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11682-1.patch, HBASE-11682.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11678) BucketCache ramCache fills heap after running a few hours
[ https://issues.apache.org/jira/browse/HBASE-11678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11678: --- Fix Version/s: 0.98.6 2.0.0 0.99.0 > BucketCache ramCache fills heap after running a few hours > - > > Key: HBASE-11678 > URL: https://issues.apache.org/jira/browse/HBASE-11678 > Project: HBase > Issue Type: Bug > Components: BlockCache >Affects Versions: 0.99.0, 0.98.5, 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: > 0001-When-we-failed-add-an-entry-failing-with-a-CacheFull.patch, > gc_over_12_hours_unevenblocks_with_lots_of_evictions.png > > > Testing BucketCache, my heap filled after running for hours. Dumping heap, > culprit is the ramCache Map in BucketCache. Tried running with more writer > threads but made no difference. Trying to figure now how our accounting is > going wonky. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11553) Abstract visibility label related services into an interface
[
https://issues.apache.org/jira/browse/HBASE-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087912#comment-14087912
]
Hadoop QA commented on HBASE-11553:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660168/HBASE-11553_V2.patch
against trunk revision .
ATTACHMENT ID: 12660168
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.http.TestHttpServerLifecycle.testStartedServerIsAlive(TestHttpServerLifecycle.java:72)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10320//console
This message is automatically generated.
> Abstract visibility label related services into an interface
>
>
> Key: HBASE-11553
> URL: https://issues.apache.org/jira/browse/HBASE-11553
> Project: HBase
> Issue Type: Improvement
> Components: security
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11553.patch, HBASE-11553.patch,
> HBASE-11553_V2.patch
>
>
> - storage and retrieval of label dictionary and authentication sets
> - marshalling and unmarshalling of visibility expression representations
> in operation attributes and cell tags
> - management of assignment of authorizations to principals
> This will allow us to introduce additional serde implementations for
> visibility expressions, for example storing as strings in some places and
> compressed/tokenized representation in others in order to support additional
> use cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11535: -- Attachment: HBASE-11535_98.patch HBASE-11535.patch > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Attachments: HBASE-11535.patch, HBASE-11535.patch, > HBASE-11535_98.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11535: -- Attachment: (was: HBASE-11535_98.patch) > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Attachments: HBASE-11535.patch, HBASE-11535.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11535: -- Attachment: HBASE-11535_98.patch > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Attachments: HBASE-11535.patch, HBASE-11535.patch, > HBASE-11535_98.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087926#comment-14087926 ] Virag Kothari commented on HBASE-11535: --- Thanks for pointing that [~jmhsieh]. Updated based on your comments and also provided a 0.98 patch > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.6 > > Attachments: HBASE-11535.patch, HBASE-11535.patch, > HBASE-11535_98.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11535: -- Fix Version/s: 0.98.6 > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.6 > > Attachments: HBASE-11535.patch, HBASE-11535.patch, > HBASE-11535_98.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11546) Backport HBASE-11059 to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087928#comment-14087928 ] Virag Kothari commented on HBASE-11546: --- Yes, end of August sounds good. > Backport HBASE-11059 to 0.98 > > > Key: HBASE-11546 > URL: https://issues.apache.org/jira/browse/HBASE-11546 > Project: HBase > Issue Type: Task >Reporter: Virag Kothari > > Discuss concerns about backporting HBASE-11059 to 0.98 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087937#comment-14087937 ] Alex Newman commented on HBASE-11665: - 1. [~devaraj] Was that comment meant for this jira 2. Other tests that do this (should I fix them?) TestDrainingServers TestAssigningManager TestAssignmentManagerOnCluster TestMasterFailover TestOpenRegionHandler TestRestartCluster TestZKOpenRegion TestZKAMOnCluster TestNameSpaceUpgrade <- Is still broken? TestSplitTransactionOnCluster TestZKLessMerge - sets it to false TestCloseRegionHandler TestOpenRegionHandler ConfigUtil - sets it to false > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11535) ReplicationPeer map is not thread safe
[
https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087936#comment-14087936
]
Hadoop QA commented on HBASE-11535:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660179/HBASE-11535_98.patch
against trunk revision .
ATTACHMENT ID: 12660179
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10322//console
This message is automatically generated.
> ReplicationPeer map is not thread safe
> --
>
> Key: HBASE-11535
> URL: https://issues.apache.org/jira/browse/HBASE-11535
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.98.4
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: 0.98.6
>
> Attachments: HBASE-11535.patch, HBASE-11535.patch,
> HBASE-11535_98.patch
>
>
> ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is
> accessed by replication source threads through getStatusOfPeer(). So, it
> needs to be thread safe.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087943#comment-14087943 ] Jimmy Xiang commented on HBASE-11665: - Can you try with patch HBASE-11611? These tests are fixed/removed. We don't need to fix them one-by-one any more. > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087945#comment-14087945 ] Jimmy Xiang commented on HBASE-11665: - BTW, HBASE-11611 is for master branch only. This test should work for branch 1. > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11658) Piped commands to hbase shell should return non-zero if shell command failed.
[ https://issues.apache.org/jira/browse/HBASE-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11658: Attachment: HBASE-11658.manual_test.log test_shell.sh HBASE-11658.patch attaching patch so that QA can take a run at it. Also attaching bash script used for manually verifying behavior and the log from a run of it. > Piped commands to hbase shell should return non-zero if shell command failed. > - > > Key: HBASE-11658 > URL: https://issues.apache.org/jira/browse/HBASE-11658 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Sean Busbey > Attachments: HBASE-11658.manual_test.log, HBASE-11658.patch, > test_shell.sh > > > See HBASE-11655. We would like the shell to return non-zero when a command > that has been piped to it fails in "scripting mode". This could be due to > invalid commands or commands issued with invalid arguments. > This would lower the barrier to entry for hbase admins so they could > effectively script some operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11535) ReplicationPeer map is not thread safe
[
https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087961#comment-14087961
]
Hadoop QA commented on HBASE-11535:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660177/HBASE-11535.patch
against trunk revision .
ATTACHMENT ID: 12660177
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
org.apache.hadoop.hbase.master.TestClockSkewDetection
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10321//console
This message is automatically generated.
> ReplicationPeer map is not thread safe
> --
>
> Key: HBASE-11535
> URL: https://issues.apache.org/jira/browse/HBASE-11535
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.98.4
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: 0.98.6
>
> Attachments: HBASE-11535.patch, HBASE-11535.patch,
> HBASE-11535_98.patch
>
>
> ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is
> accessed by replication source threads through getStatusOfPeer(). So, it
> needs to be thread safe.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11658) Piped commands to hbase shell should return non-zero if shell command failed.
[ https://issues.apache.org/jira/browse/HBASE-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11658: Status: Patch Available (was: In Progress) > Piped commands to hbase shell should return non-zero if shell command failed. > - > > Key: HBASE-11658 > URL: https://issues.apache.org/jira/browse/HBASE-11658 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Sean Busbey > Attachments: HBASE-11658.manual_test.log, HBASE-11658.patch, > test_shell.sh > > > See HBASE-11655. We would like the shell to return non-zero when a command > that has been piped to it fails in "scripting mode". This could be due to > invalid commands or commands issued with invalid arguments. > This would lower the barrier to entry for hbase admins so they could > effectively script some operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11331) [blockcache] lazy block decompression
[
https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087037#comment-14087037
]
Nick Dimiduk edited comment on HBASE-11331 at 8/6/14 6:16 PM:
--
re: lazy-decompress.02.0.pdf
Attaching some metrics after running v02 with the new flag enabled/disabled.
The test cluster is a single machine, configured with 12g heap and 0.6 for
blockcache, so slighly over 7g mem available for cache. The TestTable is snappy
compressed, written with PE --size=100. Shell {{status 'simple'}} reports
{{compressionRatio=0.2437}}. RandomRead test is run with --size=25, so with
feature enabled, we should be at ~6g of compressed blocks.
h3. client perceived perf
With flag=false, I see slightly higher throughput (~7.5k vs ~8k) and slightly
less gc (360ms vs 325ms).
h3. system load
CPU load looks slightly higher with flag=true (user time diff is negligible,
system time more noticeable, no difference in iowait).
h3. blockcache utilization
There are definitely cache evictions happening with flag=false that do not
happen with flag=true. The number of blocks cached is definitely higher with
flag=true (~17.5k vs ~21.5k).
h3. IO
No difference in reported IO requests. Probably it's the OS buffer cache to the
rescue here.
was (Author: ndimiduk):
Attaching some metrics after running v02 with the new flag enabled/disabled.
The test cluster is a single machine, configured with 12g heap and 0.6 for
blockcache, so slighly over 7g mem available for cache. The TestTable is snappy
compressed, written with PE --size=100. Shell {{status 'simple'}} reports
{{compressionRatio=0.2437}}. RandomRead test is run with --size=25, so with
feature enabled, we should be at ~6g of compressed blocks.
h3. client perceived perf
With flag=false, I see slightly higher throughput (~7.5k vs ~8k) and slightly
less gc (360ms vs 325ms).
h3. system load
CPU load looks slightly higher with flag=true (user time diff is negligible,
system time more noticeable, no difference in iowait).
h3. blockcache utilization
There are definitely cache evictions happening with flag=false that do not
happen with flag=true. The number of blocks cached is definitely higher with
flag=true (~17.5k vs ~21.5k).
h3. IO
No difference in reported IO requests. Probably it's the OS buffer cache to the
rescue here.
> [blockcache] lazy block decompression
> -
>
> Key: HBASE-11331
> URL: https://issues.apache.org/jira/browse/HBASE-11331
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch,
> HBASE-11331.02.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf,
> lazy-decompress.02.0.pdf
>
>
> Maintaining data in its compressed form in the block cache will greatly
> increase our effective blockcache size and should show a meaning improvement
> in cache hit rates in well designed applications. The idea here is to lazily
> decompress/decrypt blocks when they're consumed, rather than as soon as
> they're pulled off of disk.
> This is related to but less invasive than HBASE-8894.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-11665: Resolution: Duplicate Fix Version/s: (was: 0.99.0) Status: Resolved (was: Patch Available) DUP of HBASE-11611 > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11535: --- Resolution: Fixed Fix Version/s: 2.0.0 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1. Thanks for the patch virag! The patches had essentially 2 +1's and shouldn't affect cause problems so I've committed to 0.98, branch-1 and master. > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11535.patch, HBASE-11535.patch, > HBASE-11535_98.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11546) Backport HBASE-11059 to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11546: --- Fix Version/s: 0.98.6 Assignee: Virag Kothari Moving in to 0.98.6. Thanks Virag. > Backport HBASE-11059 to 0.98 > > > Key: HBASE-11546 > URL: https://issues.apache.org/jira/browse/HBASE-11546 > Project: HBase > Issue Type: Task >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.6 > > > Discuss concerns about backporting HBASE-11059 to 0.98 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl resolved HBASE-11667.
---
Resolution: Fixed
Committed doc update to all branches.
> Comment ClientScanner logic for NSREs.
> --
>
> Key: HBASE-11667
> URL: https://issues.apache.org/jira/browse/HBASE-11667
> Project: HBase
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
>
> Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
> HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
>
>
> We ran into an issue with Phoenix where a RegionObserver coprocessor
> intercepts a scan and returns an aggregate (in this case a count) with a fake
> row key. It turns out this does not work when the {{ClientScanner}}
> encounters NSREs, as it uses the last key it saw to reset the scanner to try
> again (which in this case would be the fake key).
> While this is arguably a rare case and one could also argue that a region
> observer just shouldn't do this... While looking at {{ClientScanner}}'s code
> I found this logic not necessary.
> A NSRE occurred because we contacted a region server with a key that it no
> longer hosts. This is the start key, so it is always correct to retry with
> this same key. That simplifies the ClientScanner logic and also make this
> sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11658) Piped commands to hbase shell should return non-zero if shell command failed.
[ https://issues.apache.org/jira/browse/HBASE-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088011#comment-14088011 ] Jonathan Hsieh commented on HBASE-11658: This is nice. I tested on trunk, currently testing on branch-1 and 0.98. [~enis], [~apurtell] do you guys want this in your respective branches? > Piped commands to hbase shell should return non-zero if shell command failed. > - > > Key: HBASE-11658 > URL: https://issues.apache.org/jira/browse/HBASE-11658 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Sean Busbey > Attachments: HBASE-11658.manual_test.log, HBASE-11658.patch, > test_shell.sh > > > See HBASE-11655. We would like the shell to return non-zero when a command > that has been piped to it fails in "scripting mode". This could be due to > invalid commands or commands issued with invalid arguments. > This would lower the barrier to entry for hbase admins so they could > effectively script some operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11658) Piped commands to hbase shell should return non-zero if shell command failed.
[
https://issues.apache.org/jira/browse/HBASE-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088032#comment-14088032
]
Hadoop QA commented on HBASE-11658:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660191/HBASE-11658.patch
against trunk revision .
ATTACHMENT ID: 12660191
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
org.apache.hadoop.hbase.master.TestClockSkewDetection
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10323//console
This message is automatically generated.
> Piped commands to hbase shell should return non-zero if shell command failed.
> -
>
> Key: HBASE-11658
> URL: https://issues.apache.org/jira/browse/HBASE-11658
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Sean Busbey
> Attachments: HBASE-11658.manual_test.log, HBASE-11658.patch,
> test_shell.sh
>
>
> See HBASE-11655. We would like the shell to return non-zero when a command
> that has been piped to it fails in "scripting mode". This could be due to
> invalid commands or commands issued with invalid arguments.
> This would lower the barrier to entry for hbase admins so they could
> effectively script some operations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11686) Shell code should create a binding / irb workspace instead of polluting the root namespace
Sean Busbey created HBASE-11686:
---
Summary: Shell code should create a binding / irb workspace
instead of polluting the root namespace
Key: HBASE-11686
URL: https://issues.apache.org/jira/browse/HBASE-11686
Project: HBase
Issue Type: Improvement
Reporter: Sean Busbey
Priority: Minor
Right now, the shell builds a list of commands and then injects them into the
root exectution's context
bin/hirb.rb
{code}
# Add commands to this namespace
@shell.export_commands(self)
{code}
hbase-shell/src/main/ruby/shell.rb
{code}
def export_commands(where)
::Shell.commands.keys.each do |cmd|
# here where is the IRB namespace
# this method just adds the call to the specified command
# which just references back to 'this' shell object
# a decently extensible way to add commands
where.send :instance_eval, <<-EOF
def #{cmd}(*args)
ret = @shell.command('#{cmd}', *args)
puts
return ret
end
EOF
end
end
{code}
This is an unclean abstraction. For one, it requires that there be an instance
variable in the main namespace called '@shell' without making that clear in the
docs. Additionally, it complicates maintenance by breaking isolation.
We should update things so that shell can provide a binding for eval or a
workspace for IRB execution and then use it directly when we construct our IRB
session.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11686) Shell code should create a binding / irb workspace instead of polluting the root namespace
[
https://issues.apache.org/jira/browse/HBASE-11686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088049#comment-14088049
]
Sean Busbey commented on HBASE-11686:
-
an example of the complication that's just fall out of the main namespace
pollution: you can't write tests that work on the exported command unless you
know to construct an object that has a member variable '@shell'
> Shell code should create a binding / irb workspace instead of polluting the
> root namespace
> --
>
> Key: HBASE-11686
> URL: https://issues.apache.org/jira/browse/HBASE-11686
> Project: HBase
> Issue Type: Improvement
>Reporter: Sean Busbey
>Priority: Minor
>
> Right now, the shell builds a list of commands and then injects them into the
> root exectution's context
> bin/hirb.rb
> {code}
> # Add commands to this namespace
> @shell.export_commands(self)
> {code}
> hbase-shell/src/main/ruby/shell.rb
> {code}
> def export_commands(where)
> ::Shell.commands.keys.each do |cmd|
> # here where is the IRB namespace
> # this method just adds the call to the specified command
> # which just references back to 'this' shell object
> # a decently extensible way to add commands
> where.send :instance_eval, <<-EOF
> def #{cmd}(*args)
> ret = @shell.command('#{cmd}', *args)
> puts
> return ret
> end
> EOF
> end
> end
> {code}
> This is an unclean abstraction. For one, it requires that there be an
> instance variable in the main namespace called '@shell' without making that
> clear in the docs. Additionally, it complicates maintenance by breaking
> isolation.
> We should update things so that shell can provide a binding for eval or a
> workspace for IRB execution and then use it directly when we construct our
> IRB session.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11687) Cancel postOpenDeployTasks if region opening is cancelled
Jimmy Xiang created HBASE-11687: --- Summary: Cancel postOpenDeployTasks if region opening is cancelled Key: HBASE-11687 URL: https://issues.apache.org/jira/browse/HBASE-11687 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 1.0.0 With ZK-less region assignment, if region opening is in postOpenDeployTasks step and the opening is cancelled, the region server will abort because it can't report the transition to the master. We should cancel postOpenDeployTasks instead. At least, there is no need to abort. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088075#comment-14088075 ] Alex Newman commented on HBASE-11665: - Oh I checked to make sure we don't need this in branch-1 BTW. It passes on my laptop! > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11658) Piped commands to hbase shell should return non-zero if shell command failed.
[ https://issues.apache.org/jira/browse/HBASE-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11658: Attachment: HBASE_11658-v2.patch Attaching v2 from the RB. I took a look at the QA results. Since this patch only touches the shell ruby code, I don't see how it could have caused the test failures in hbase-server or the change in findbugs warnings. > Piped commands to hbase shell should return non-zero if shell command failed. > - > > Key: HBASE-11658 > URL: https://issues.apache.org/jira/browse/HBASE-11658 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Sean Busbey > Attachments: HBASE-11658.manual_test.log, HBASE-11658.patch, > HBASE_11658-v2.patch, test_shell.sh > > > See HBASE-11655. We would like the shell to return non-zero when a command > that has been piped to it fails in "scripting mode". This could be due to > invalid commands or commands issued with invalid arguments. > This would lower the barrier to entry for hbase admins so they could > effectively script some operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11567) Write bulk load events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088082#comment-14088082 ] Alex Newman commented on HBASE-11567: - I think we should wait to commit this until the reader side is done. I think figuring out those tests are important for the acceptability of this patch. That being said, I hope this is a useful first step for whoever (most likely me) has to do it. > Write bulk load events to WAL > - > > Key: HBASE-11567 > URL: https://issues.apache.org/jira/browse/HBASE-11567 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alex Newman > Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch > > > Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and > region open/close (HBASE-11512) , we should persist bulk load events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11686) Shell code should create a binding / irb workspace instead of polluting the root namespace
[
https://issues.apache.org/jira/browse/HBASE-11686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-11686:
Component/s: shell
> Shell code should create a binding / irb workspace instead of polluting the
> root namespace
> --
>
> Key: HBASE-11686
> URL: https://issues.apache.org/jira/browse/HBASE-11686
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Sean Busbey
>Priority: Minor
>
> Right now, the shell builds a list of commands and then injects them into the
> root exectution's context
> bin/hirb.rb
> {code}
> # Add commands to this namespace
> @shell.export_commands(self)
> {code}
> hbase-shell/src/main/ruby/shell.rb
> {code}
> def export_commands(where)
> ::Shell.commands.keys.each do |cmd|
> # here where is the IRB namespace
> # this method just adds the call to the specified command
> # which just references back to 'this' shell object
> # a decently extensible way to add commands
> where.send :instance_eval, <<-EOF
> def #{cmd}(*args)
> ret = @shell.command('#{cmd}', *args)
> puts
> return ret
> end
> EOF
> end
> end
> {code}
> This is an unclean abstraction. For one, it requires that there be an
> instance variable in the main namespace called '@shell' without making that
> clear in the docs. Additionally, it complicates maintenance by breaking
> isolation.
> We should update things so that shell can provide a binding for eval or a
> workspace for IRB execution and then use it directly when we construct our
> IRB session.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11678) BucketCache ramCache fills heap after running a few hours
[ https://issues.apache.org/jira/browse/HBASE-11678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11678: -- Attachment: 11678v2.txt Managed to get a test in there for each of the modes that can happen adding item to BucketCache. Lets see what the findbugs issues are. Test failures look unrelated. > BucketCache ramCache fills heap after running a few hours > - > > Key: HBASE-11678 > URL: https://issues.apache.org/jira/browse/HBASE-11678 > Project: HBase > Issue Type: Bug > Components: BlockCache >Affects Versions: 0.99.0, 0.98.5, 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: > 0001-When-we-failed-add-an-entry-failing-with-a-CacheFull.patch, 11678v2.txt, > gc_over_12_hours_unevenblocks_with_lots_of_evictions.png > > > Testing BucketCache, my heap filled after running for hours. Dumping heap, > culprit is the ramCache Map in BucketCache. Tried running with more writer > threads but made no difference. Trying to figure now how our accounting is > going wonky. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11603) Apply version of HADOOP-8027 to our JMXJsonServlet
[
https://issues.apache.org/jira/browse/HBASE-11603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11603:
--
Attachment: 0001-HBASE-11603-Apply-version-of-HADOOP-8027-to-our-JMXJ.patch
Thanks [~ndimiduk]
Testing patch shows need to be more aggressive still.
> Apply version of HADOOP-8027 to our JMXJsonServlet
> --
>
> Key: HBASE-11603
> URL: https://issues.apache.org/jira/browse/HBASE-11603
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments:
> 0001-HBASE-11603-Apply-version-of-HADOOP-8027-to-our-JMXJ.patch,
> 0001-HBASE-11603-Apply-version-of-HADOOP-8027-to-our-JMXJ.patch, 11603.txt,
> 11603v2.txt
>
>
> Running master, I see below when I poll /jmx servlet:
> {code}
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1345)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> Caused by: java.lang.UnsupportedOperationException: CollectionUsage threshold
> is not supported
> at
> sun.management.MemoryPoolImpl.isCollectionUsageThresholdExceeded(MemoryPoolImpl.java:242)
> at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at
> com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)
> at
> com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)
> at
> com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)
> at
> com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at javax.management.StandardMBean.getAttribute(StandardMBean.java:372)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> {code}
> This has been seen before
[jira] [Updated] (HBASE-11603) Apply version of HADOOP-8027 to our JMXJsonServlet
[
https://issues.apache.org/jira/browse/HBASE-11603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11603:
--
Status: Patch Available (was: Reopened)
> Apply version of HADOOP-8027 to our JMXJsonServlet
> --
>
> Key: HBASE-11603
> URL: https://issues.apache.org/jira/browse/HBASE-11603
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments:
> 0001-HBASE-11603-Apply-version-of-HADOOP-8027-to-our-JMXJ.patch,
> 0001-HBASE-11603-Apply-version-of-HADOOP-8027-to-our-JMXJ.patch, 11603.txt,
> 11603v2.txt
>
>
> Running master, I see below when I poll /jmx servlet:
> {code}
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1345)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> Caused by: java.lang.UnsupportedOperationException: CollectionUsage threshold
> is not supported
> at
> sun.management.MemoryPoolImpl.isCollectionUsageThresholdExceeded(MemoryPoolImpl.java:242)
> at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at
> com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)
> at
> com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)
> at
> com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)
> at
> com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at javax.management.StandardMBean.getAttribute(StandardMBean.java:372)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> {code}
> This has been seen before over in HADOOP-8027. We get it since we brought
> jetty server local from hadoop. I don't think it in branc
[jira] [Updated] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11331: - Attachment: lazy-decompress.02.1.pdf re: lazy-decompress.02.1.pdf Attaching some metrics after running v02 with less os page cache and feature enabled/disabled. This time the total RS heap is increased to 20g, so the effective blockcache size is ~12g. The machine has 24g of RAM, so this squeezes the OS-available memory down to ~4g -- when the blockcache is full. I've added blockcache size and OS page cache to the charts Same table, same compression ratio, same PE randomRead test, this time run with --size=45, so I expect ~11g of compressed blocks. I expected this to be an IO-bound workload with flag=false but to be cache-at-capacity for flag=true. The massive evictions look to me like the blockcache heapsize calculation is based on the decompressed size, not compressed size. Updating patch. > [blockcache] lazy block decompression > - > > Key: HBASE-11331 > URL: https://issues.apache.org/jira/browse/HBASE-11331 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch, > HBASE-11331.02.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf, > lazy-decompress.02.0.pdf, lazy-decompress.02.1.pdf > > > Maintaining data in its compressed form in the block cache will greatly > increase our effective blockcache size and should show a meaning improvement > in cache hit rates in well designed applications. The idea here is to lazily > decompress/decrypt blocks when they're consumed, rather than as soon as > they're pulled off of disk. > This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088136#comment-14088136 ] stack commented on HBASE-11331: --- Any chance of our fixing compressor reuse. As you say, I'd imagine it'd mess up any possibility of nice numbers when this feature enabled. I'm game to rerun test when you say its ready. That a fancy graph tool on front a tsdb? [~ndimiduk]? Nice graphs. > [blockcache] lazy block decompression > - > > Key: HBASE-11331 > URL: https://issues.apache.org/jira/browse/HBASE-11331 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch, > HBASE-11331.02.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf, > lazy-decompress.02.0.pdf, lazy-decompress.02.1.pdf > > > Maintaining data in its compressed form in the block cache will greatly > increase our effective blockcache size and should show a meaning improvement > in cache hit rates in well designed applications. The idea here is to lazily > decompress/decrypt blocks when they're consumed, rather than as soon as > they're pulled off of disk. > This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11688) Add histograms of key attributes to server front pages
stack created HBASE-11688: - Summary: Add histograms of key attributes to server front pages Key: HBASE-11688 URL: https://issues.apache.org/jira/browse/HBASE-11688 Project: HBase Issue Type: Improvement Reporter: stack Priority: Minor Messing with histograms of late, seems a pity we don't show in UI histograms of basic recent events and metrics. In past we've tried to keep history in a history table and we've talked of bringing this back as data source for being able to mine what has been happening of late and in the recent past but meantime, we could keep running in-memory histograms of recent history and show cute little histo-diagrams of hits, compactions, flushes... Would be nice little project with a high profile. It'd not be a replacement for cluster monitoring but more just an at-a-glance high-level dashboard that gives a clue on "...what has my cluster been up to of late?" -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11688) Add histograms of key attributes to server front pages
[ https://issues.apache.org/jira/browse/HBASE-11688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11688: -- Component/s: monitoring > Add histograms of key attributes to server front pages > -- > > Key: HBASE-11688 > URL: https://issues.apache.org/jira/browse/HBASE-11688 > Project: HBase > Issue Type: Improvement > Components: monitoring >Reporter: stack >Priority: Minor > Labels: beginner > > Messing with histograms of late, seems a pity we don't show in UI histograms > of basic recent events and metrics. In past we've tried to keep history in a > history table and we've talked of bringing this back as data source for being > able to mine what has been happening of late and in the recent past but > meantime, we could keep running in-memory histograms of recent history and > show cute little histo-diagrams of hits, compactions, flushes... Would be > nice little project with a high profile. It'd not be a replacement for > cluster monitoring but more just an at-a-glance high-level dashboard that > gives a clue on "...what has my cluster been up to of late?" -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11567) Write bulk load events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088170#comment-14088170 ] stack commented on HBASE-11567: --- Took a quick look. Looks good to me. We could commit this but agree probably better if read-side goes in around same time. > Write bulk load events to WAL > - > > Key: HBASE-11567 > URL: https://issues.apache.org/jira/browse/HBASE-11567 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alex Newman > Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch > > > Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and > region open/close (HBASE-11512) , we should persist bulk load events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11567) Write bulk load events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088177#comment-14088177 ] Alex Newman commented on HBASE-11567: - Daww thanks stack. One question I have is, currently I am logging the request by the user for HBase HFile bulk loading. That being said, does it make sense to have a different WAL entry when it completes successfully? > Write bulk load events to WAL > - > > Key: HBASE-11567 > URL: https://issues.apache.org/jira/browse/HBASE-11567 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alex Newman > Attachments: HBASE-11567-v1.patch, HBASE-11567-v2.patch > > > Similar to writing flush (HBASE-11511), compaction(HBASE-2231) to WAL and > region open/close (HBASE-11512) , we should persist bulk load events to WAL. > This is especially important for secondary region replicas, since we can use > this information to pick up primary regions' files from secondary replicas. > A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11658) Piped commands to hbase shell should return non-zero if shell command failed.
[ https://issues.apache.org/jira/browse/HBASE-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088176#comment-14088176 ] stack commented on HBASE-11658: --- Skimmed. It looks great. Failures are infrastructure issues. See https://issues.apache.org/jira/browse/BUILDS-4 > Piped commands to hbase shell should return non-zero if shell command failed. > - > > Key: HBASE-11658 > URL: https://issues.apache.org/jira/browse/HBASE-11658 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Sean Busbey > Attachments: HBASE-11658.manual_test.log, HBASE-11658.patch, > HBASE_11658-v2.patch, test_shell.sh > > > See HBASE-11655. We would like the shell to return non-zero when a command > that has been piped to it fails in "scripting mode". This could be due to > invalid commands or commands issued with invalid arguments. > This would lower the barrier to entry for hbase admins so they could > effectively script some operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088187#comment-14088187 ] stack commented on HBASE-11611: --- Skimmed. [~jxiang] This looks to be candidate for best patch of 2014! Its great. Can you get hadoopqa to apply it? I think we just apply if it passes. > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088190#comment-14088190 ] Hudson commented on HBASE-11535: SUCCESS: Integrated in HBase-0.98 #438 (See [https://builds.apache.org/job/HBase-0.98/438/]) HBASE-11535 ReplicationPeer map is not thread safe (Virag Kothari) (jon: rev c8812737bb53016ec18738061b9c2a18ca8c0ad8) * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11535.patch, HBASE-11535.patch, > HBASE-11535_98.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088193#comment-14088193 ] Nick Dimiduk commented on HBASE-11331: -- bq. Any chance of our fixing compressor reuse. I'll look into it. If anything, it'll be a separate ticket. bq. I'm game to rerun test when you say its ready. Hold off on rerunning until I verify/refute my hypothesis on the blockcache heapsize stuff. bq. Nice graphs Awe shucks, thanks boss. This is Grafana running on top of OpenTSDB. No offense, but I find it nicer to use these dashboards than the gnuplot stuff we get otherwise. I'm still learning it, but I'll attach the dashboard json file I used to make the latest figures. > [blockcache] lazy block decompression > - > > Key: HBASE-11331 > URL: https://issues.apache.org/jira/browse/HBASE-11331 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch, > HBASE-11331.02.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf, > lazy-decompress.02.0.pdf, lazy-decompress.02.1.pdf > > > Maintaining data in its compressed form in the block cache will greatly > increase our effective blockcache size and should show a meaning improvement > in cache hit rates in well designed applications. The idea here is to lazily > decompress/decrypt blocks when they're consumed, rather than as soon as > they're pulled off of disk. > This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088191#comment-14088191 ] stack commented on HBASE-11665: --- bq. Oh I checked to make sure we don't need this in branch-1 Sorry [~posix4e] We need this patch or not? Thanks. [~jxiang] this flag is set true in branch-1 already? We don't need set it in the test? > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11331: - Attachment: lazy-decompress.02.1.json Attach lazy-decompress.02.1.json, used with grafana 1.7rc for the legend aliasing. > [blockcache] lazy block decompression > - > > Key: HBASE-11331 > URL: https://issues.apache.org/jira/browse/HBASE-11331 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch, > HBASE-11331.02.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf, > lazy-decompress.02.0.pdf, lazy-decompress.02.1.json, lazy-decompress.02.1.pdf > > > Maintaining data in its compressed form in the block cache will greatly > increase our effective blockcache size and should show a meaning improvement > in cache hit rates in well designed applications. The idea here is to lazily > decompress/decrypt blocks when they're consumed, rather than as soon as > they're pulled off of disk. > This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088194#comment-14088194 ] Alex Newman commented on HBASE-11665: - We will not need it in branch-1 > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11665) TestRegionReplicas doesn't work for me
[ https://issues.apache.org/jira/browse/HBASE-11665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088210#comment-14088210 ] Jimmy Xiang commented on HBASE-11665: - The flag is set to true by default in branch-1. So we don't need set it in the test. > TestRegionReplicas doesn't work for me > -- > > Key: HBASE-11665 > URL: https://issues.apache.org/jira/browse/HBASE-11665 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: Alex Newman > Fix For: 2.0.0 > > Attachments: HBASE-11665-v1.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11660) Make WAL reader follow contract for java.io.InputStream.available()
[ https://issues.apache.org/jira/browse/HBASE-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11660: --- Affects Version/s: 0.98.6 2.0.0 0.99.0 > Make WAL reader follow contract for java.io.InputStream.available() > --- > > Key: HBASE-11660 > URL: https://issues.apache.org/jira/browse/HBASE-11660 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.99.0, 2.0.0, 0.98.6 >Reporter: Eric Hanson >Priority: Minor > Attachments: hbase-11660.01.patch > > > In the process of building support to running HBase on Microsoft Azure > HDInsight, I hit an issue in the HBase WAL reading process that took a lot of > time to debug. The WAL reading code depends on available() for the log > InputStream never returing 0 until end of file. This is not the same as the > contract in java.io.InputStream for available. > To prevent future grief for others that may want to port HBase onto storage > systems other than HDFS, I propose to change the HBase WAL reader so it does > not assume that EOF has been reached when available() == 0. It instead would > treat available only as described in InputStream, i.e. available() is merely > the number of bytes that could be read from the stream without blocking. That > could be 0 even before EOF. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088229#comment-14088229
]
Andrew Purtell commented on HBASE-11685:
This change does not preserve the old behavior where we avoid taking refCount
negative. A concern? We should warn loudly if this goes negative at least I
think.
{code}
/**
* Decrement this client's reference count.
*/
void decCount() {
- if (refCount > 0) {
---refCount;
- }
+ refCount.decrementAndGet();
}
{code}
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11685-trunk-v1.diff, HBASE-11685-trunk-v2.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11546) Backport HBASE-11059 (ZK-less region assignment) to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11546: --- Summary: Backport HBASE-11059 (ZK-less region assignment) to 0.98 (was: Backport HBASE-11059 to 0.98) > Backport HBASE-11059 (ZK-less region assignment) to 0.98 > > > Key: HBASE-11546 > URL: https://issues.apache.org/jira/browse/HBASE-11546 > Project: HBase > Issue Type: Task >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.98.6 > > > Discuss concerns about backporting HBASE-11059 to 0.98 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11678) BucketCache ramCache fills heap after running a few hours
[
https://issues.apache.org/jira/browse/HBASE-11678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088234#comment-14088234
]
Nick Dimiduk commented on HBASE-11678:
--
patch v2:
{noformat}
+"; adjust BucketCache sizes " + CacheConfig.BUCKET_CACHE_BUCKETS_KEY +
+" to accomodate if size seems reasonable and you want it cached.");
{noformat}
Nice, I like it.
{noformat}
+private volatile boolean writerEnabled = true;
{noformat}
Why mark this volatile?
{noformat}
- if (cacheEnabled
- && (now - ioErrorStartTime) > this.ioErrorsTolerationDuration) {
-LOG.error("IO errors duration time has exceeded "
-+ ioErrorsTolerationDuration
-+ "ms, disabing cache, please check your IOEngine");
+ if (cacheEnabled && (now - ioErrorStartTime) >
this.ioErrorsTolerationDuration) {
+LOG.error("IO errors duration time has exceeded " +
ioErrorsTolerationDuration +
+ "ms, disabing cache, please check your IOEngine");
disableCache();
{noformat}
I know there's no new functionality here, but shouldn't this be an RS-abort
kind of offense?
{noformat}
+this.bc = new BucketCache("heap", capacity, 1, new int [] {1},
writerThreadsCount,
{noformat}
nit: isn't "heap" and friends in an Enum somewhere? ... Oh, there's not.
Probably should be.
Nice new tests. Are you able to verify that testTooBigEntry fails on current
master? Maybe too much refactoring is necessary to confirm.
+1
> BucketCache ramCache fills heap after running a few hours
> -
>
> Key: HBASE-11678
> URL: https://issues.apache.org/jira/browse/HBASE-11678
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Affects Versions: 0.99.0, 0.98.5, 2.0.0
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments:
> 0001-When-we-failed-add-an-entry-failing-with-a-CacheFull.patch, 11678v2.txt,
> gc_over_12_hours_unevenblocks_with_lots_of_evictions.png
>
>
> Testing BucketCache, my heap filled after running for hours. Dumping heap,
> culprit is the ramCache Map in BucketCache. Tried running with more writer
> threads but made no difference. Trying to figure now how our accounting is
> going wonky.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088241#comment-14088241
]
Hudson commented on HBASE-11667:
FAILURE: Integrated in HBase-0.94-JDK7 #167 (See
[https://builds.apache.org/job/HBase-0.94-JDK7/167/])
HBASE-11667 Comment ClientScanner logic for NSREs. (larsh: rev
c51e19afec1925de7769c24e0dbd2bfcf90a184e)
* src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
> Comment ClientScanner logic for NSREs.
> --
>
> Key: HBASE-11667
> URL: https://issues.apache.org/jira/browse/HBASE-11667
> Project: HBase
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
>
> Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
> HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
>
>
> We ran into an issue with Phoenix where a RegionObserver coprocessor
> intercepts a scan and returns an aggregate (in this case a count) with a fake
> row key. It turns out this does not work when the {{ClientScanner}}
> encounters NSREs, as it uses the last key it saw to reset the scanner to try
> again (which in this case would be the fake key).
> While this is arguably a rare case and one could also argue that a region
> observer just shouldn't do this... While looking at {{ClientScanner}}'s code
> I found this logic not necessary.
> A NSRE occurred because we contacted a region server with a key that it no
> longer hosts. This is the start key, so it is always correct to retry with
> this same key. That simplifies the ClientScanner logic and also make this
> sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11658) Piped commands to hbase shell should return non-zero if shell command failed.
[ https://issues.apache.org/jira/browse/HBASE-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088262#comment-14088262 ] Jonathan Hsieh commented on HBASE-11658: lgtm -- applies to branch-1 and 0.98 and looks like it works on both. waiting for their feedback before I commit to master and those branches. > Piped commands to hbase shell should return non-zero if shell command failed. > - > > Key: HBASE-11658 > URL: https://issues.apache.org/jira/browse/HBASE-11658 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Sean Busbey > Attachments: HBASE-11658.manual_test.log, HBASE-11658.patch, > HBASE_11658-v2.patch, test_shell.sh > > > See HBASE-11655. We would like the shell to return non-zero when a command > that has been piped to it fails in "scripting mode". This could be due to > invalid commands or commands issued with invalid arguments. > This would lower the barrier to entry for hbase admins so they could > effectively script some operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088268#comment-14088268
]
Hudson commented on HBASE-11667:
FAILURE: Integrated in HBase-TRUNK #5375 (See
[https://builds.apache.org/job/HBase-TRUNK/5375/])
HBASE-11667 Comment ClientScanner logic for NSREs. (larsh: rev
25e6baee424602942afbe40b01ea0b4cff1ea9b1)
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
> Comment ClientScanner logic for NSREs.
> --
>
> Key: HBASE-11667
> URL: https://issues.apache.org/jira/browse/HBASE-11667
> Project: HBase
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
>
> Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
> HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
>
>
> We ran into an issue with Phoenix where a RegionObserver coprocessor
> intercepts a scan and returns an aggregate (in this case a count) with a fake
> row key. It turns out this does not work when the {{ClientScanner}}
> encounters NSREs, as it uses the last key it saw to reset the scanner to try
> again (which in this case would be the fake key).
> While this is arguably a rare case and one could also argue that a region
> observer just shouldn't do this... While looking at {{ClientScanner}}'s code
> I found this logic not necessary.
> A NSRE occurred because we contacted a region server with a key that it no
> longer hosts. This is the start key, so it is always correct to retry with
> this same key. That simplifies the ClientScanner logic and also make this
> sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11535) ReplicationPeer map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088267#comment-14088267 ] Hudson commented on HBASE-11535: FAILURE: Integrated in HBase-TRUNK #5375 (See [https://builds.apache.org/job/HBase-TRUNK/5375/]) HBASE-11535 ReplicationPeer map is not thread safe (Virag Kothari) (jmhsieh: rev 6dee406bf359eec31883cdb2714399a8161a4c26) * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java > ReplicationPeer map is not thread safe > -- > > Key: HBASE-11535 > URL: https://issues.apache.org/jira/browse/HBASE-11535 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.4 >Reporter: Virag Kothari >Assignee: Virag Kothari > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11535.patch, HBASE-11535.patch, > HBASE-11535_98.patch > > > ReplicationPeersZKImpl has a map of Peer id to ReplicationPeer which is > accessed by replication source threads through getStatusOfPeer(). So, it > needs to be thread safe. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
[ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088271#comment-14088271 ] Andrew Purtell commented on HBASE-11331: bq. Grafana running on top of OpenTSDB Very nice. Is there a way to autogen the JSON? Pardon for hijacking the issue. > [blockcache] lazy block decompression > - > > Key: HBASE-11331 > URL: https://issues.apache.org/jira/browse/HBASE-11331 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch, > HBASE-11331.02.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf, > lazy-decompress.02.0.pdf, lazy-decompress.02.1.json, lazy-decompress.02.1.pdf > > > Maintaining data in its compressed form in the block cache will greatly > increase our effective blockcache size and should show a meaning improvement > in cache hit rates in well designed applications. The idea here is to lazily > decompress/decrypt blocks when they're consumed, rather than as soon as > they're pulled off of disk. > This is related to but less invasive than HBASE-8894. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11681) Update and move doc about disabling the WAL
[ https://issues.apache.org/jira/browse/HBASE-11681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11681: --- Affects Version/s: 2.0.0 0.99.0 > Update and move doc about disabling the WAL > --- > > Key: HBASE-11681 > URL: https://issues.apache.org/jira/browse/HBASE-11681 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.99.0, 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11681.patch > > > Move the docs about disabling the WAL from the Performance section to the WAL > section, point to the new location, and add info about getDurability and > setDurability methods. Leave the big fat warnings about data loss in both the > Performance and WAL sections. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11681) Update and move doc about disabling the WAL
[ https://issues.apache.org/jira/browse/HBASE-11681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11681: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) lgtm, committing to branch-1 and master. > Update and move doc about disabling the WAL > --- > > Key: HBASE-11681 > URL: https://issues.apache.org/jira/browse/HBASE-11681 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.99.0, 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11681.patch > > > Move the docs about disabling the WAL from the Performance section to the WAL > section, point to the new location, and add info about getDurability and > setDurability methods. Leave the big fat warnings about data loss in both the > Performance and WAL sections. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11611: Attachment: hbase-11611_v2.patch > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch, > hbase-11611_v2.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088278#comment-14088278 ] Jimmy Xiang commented on HBASE-11611: - Attached v2. Hopefully it's ok with hadoopqa. Thanks. > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch, > hbase-11611_v2.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (HBASE-11673) TestIOFencing#testFencingAroundCompactionAfterWALSync fails
[
https://issues.apache.org/jira/browse/HBASE-11673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reopened HBASE-11673:
The test fails in HBase-TRUNK build #5375
Mind taking a look ?
> TestIOFencing#testFencingAroundCompactionAfterWALSync fails
> ---
>
> Key: HBASE-11673
> URL: https://issues.apache.org/jira/browse/HBASE-11673
> Project: HBase
> Issue Type: Test
>Reporter: Qiang Tian
>Assignee: Sergey Soldatov
> Fix For: 2.0.0
>
> Attachments: HBASE_11673-v1.patch
>
>
> got several test failure on the latest build:
> {quote}
> [tianq@bdvm101 surefire-reports]$ ls -1t|grep "Tests run" * |grep "<<<
> FAILURE"
> org.apache.hadoop.hbase.client.TestReplicasClient.txt:Tests run: 1, Failures:
> 0, Errors: 1, Skipped: 0, Time elapsed: 38.706 sec <<< FAILURE!
> org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas.txt:Tests
> run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 30.669 sec <<<
> FAILURE!
> org.apache.hadoop.hbase.regionserver.TestRegionReplicas.txt:Tests run: 1,
> Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 39.113 sec <<< FAILURE!
> org.apache.hadoop.hbase.TestIOFencing.txt:Tests run: 2, Failures: 1, Errors:
> 0, Skipped: 0, Time elapsed: 177.071 sec <<< FAILURE!
> {quote}
> the first one:
> {quote}
> type="java.lang.AssertionError">java.lang.AssertionError: Timed out waiting
> for the region to flush
> >-at org.junit.Assert.fail(Assert.java:88)
> >-at org.junit.Assert.assertTrue(Assert.java:41)
> >-at org.apache.hadoop.hbase.TestIOFencing.doTest(TestIOFencing.java:291)
> >-at
> >org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompactionAfterWALSync(TestIOFencing.java:236)
> >-at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >-at
> >sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >-at
> >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >-at java.lang.reflect.Method.invoke(Method.java:606)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11611) Clean up ZK-based region assignment
[ https://issues.apache.org/jira/browse/HBASE-11611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088277#comment-14088277 ] Alex Newman commented on HBASE-11611: - That much red makes me shed tears of joy. > Clean up ZK-based region assignment > --- > > Key: HBASE-11611 > URL: https://issues.apache.org/jira/browse/HBASE-11611 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 2.0.0 > > Attachments: hbase-11611.patch, hbase-11611_v1.patch, > hbase-11611_v2.patch > > > We can clean up the ZK-based region assignment code and use the ZK-less one > in the master branch, to make the code easier to understand and maintain. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11681) Update and move doc about disabling the WAL
[ https://issues.apache.org/jira/browse/HBASE-11681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088284#comment-14088284 ] Jonathan Hsieh commented on HBASE-11681: thanks misty! > Update and move doc about disabling the WAL > --- > > Key: HBASE-11681 > URL: https://issues.apache.org/jira/browse/HBASE-11681 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.99.0, 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11681.patch > > > Move the docs about disabling the WAL from the Performance section to the WAL > section, point to the new location, and add info about getDurability and > setDurability methods. Leave the big fat warnings about data loss in both the > Performance and WAL sections. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4593) Design and document the official procedure for posting patches, commits, commit messages, etc. to smooth process and make integration with tools easier
[
https://issues.apache.org/jira/browse/HBASE-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088291#comment-14088291
]
Misty Stanley-Jones commented on HBASE-4593:
>From an email from [~apurtell]
{code}
-- Forwarded message --
From: Andrew Purtell
Date: Fri, Aug 1, 2014 at 10:32 AM
Subject: Friendly reminder to use git cherry-pick and git rebase instead of git
merge
To: "d...@hbase.apache.org"
We previously agreed to avoid merge commits appearing in our git history.
Found a merge commit on 0.98 branch.
Thanks!
{code}
> Design and document the official procedure for posting patches, commits,
> commit messages, etc. to smooth process and make integration with tools easier
> ---
>
> Key: HBASE-4593
> URL: https://issues.apache.org/jira/browse/HBASE-4593
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Jonathan Gray
>Assignee: Misty Stanley-Jones
>
> I have been building a tool (currently called reposync) to help me keep the
> internal FB hbase-92-based branch up-to-date with the public branches.
> Various inconsistencies in our process has made it difficult to automate a
> lot of this stuff.
> I'd like to work with everyone to come up with the official best practices
> and stick to it.
> I welcome all suggestions. Among some of the things I'd like to nail down:
> - Commit message format
> - Best practice and commit message format for multiple commits
> - Multiple commits per jira vs. jira per commit, what are the exceptions and
> when
> - Affects vs. Fix versions
> - Potential usage of [tags] in commit messages for things like book, scripts,
> shell... maybe even whatever is in the components field?
> - Increased usage of JIRA tags or labels to mark exactly which repos a JIRA
> has been committed to (potentially even internal repos? ways for a tool to
> keep track in JIRA?)
> We also need to be more strict about some things if we want to follow Apache
> guidelines. For example, all final versions of a patch must be attached to
> JIRA so that the author properly assigns it to Apache.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4593) Design and document the official procedure for posting patches, commits, commit messages, etc. to smooth process and make integration with tools easier
[ https://issues.apache.org/jira/browse/HBASE-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088293#comment-14088293 ] Misty Stanley-Jones commented on HBASE-4593: What about ReviewBoard? Can we do better than http://hbase.apache.org/book.html#reviewboard? What is "larger patches"? Can someone describe the workflow of ReviewBoard for me? I have never used it. > Design and document the official procedure for posting patches, commits, > commit messages, etc. to smooth process and make integration with tools easier > --- > > Key: HBASE-4593 > URL: https://issues.apache.org/jira/browse/HBASE-4593 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Jonathan Gray >Assignee: Misty Stanley-Jones > > I have been building a tool (currently called reposync) to help me keep the > internal FB hbase-92-based branch up-to-date with the public branches. > Various inconsistencies in our process has made it difficult to automate a > lot of this stuff. > I'd like to work with everyone to come up with the official best practices > and stick to it. > I welcome all suggestions. Among some of the things I'd like to nail down: > - Commit message format > - Best practice and commit message format for multiple commits > - Multiple commits per jira vs. jira per commit, what are the exceptions and > when > - Affects vs. Fix versions > - Potential usage of [tags] in commit messages for things like book, scripts, > shell... maybe even whatever is in the components field? > - Increased usage of JIRA tags or labels to mark exactly which repos a JIRA > has been committed to (potentially even internal repos? ways for a tool to > keep track in JIRA?) > We also need to be more strict about some things if we want to follow Apache > guidelines. For example, all final versions of a patch must be attached to > JIRA so that the author properly assigns it to Apache. -- This message was sent by Atlassian JIRA (v6.2#6252)
