[jira] [Commented] (HBASE-15075) Allow region split request to carry identification information
[
https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103100#comment-15103100
]
Hadoop QA commented on HBASE-15075:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 38s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 48s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 29s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 49s
{color} | {color:red} hbase-server in master has 82 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 20s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 54s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 20s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 37s
{color} | {color:green} hbase-protocol in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 4s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 100m
[jira] [Commented] (HBASE-14512) Cache UGI groups
[ https://issues.apache.org/jira/browse/HBASE-14512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103161#comment-15103161 ] Hudson commented on HBASE-14512: FAILURE: Integrated in HBase-0.98-matrix #287 (See [https://builds.apache.org/job/HBase-0.98-matrix/287/]) Amend HBASE-14512 Cache UGI groups (apurtell: rev d5f8300c082a75ce8edbbe08b66f077e7d663a4a) * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java > Cache UGI groups > > > Key: HBASE-14512 > URL: https://issues.apache.org/jira/browse/HBASE-14512 > Project: HBase > Issue Type: Bug > Components: Performance, security >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14512-addendum-0.98.patch, HBASE-14512-v1.patch, > HBASE-14512-v2.patch, HBASE-14512-v3.patch, HBASE-14512-v4.patch, > HBASE-14512.patch > > > Right now every call gets a new User object. > We should keep the same user for the life of a connection. We should also > cache the group names. However we can't cache the groups for forever as that > would mean groups don't get refreshed every 5 mins. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103159#comment-15103159 ] Hudson commented on HBASE-14976: FAILURE: Integrated in HBase-0.98-matrix #287 (See [https://builds.apache.org/job/HBase-0.98-matrix/287/]) Amend HBASE-14976 Add RPC call queues to the web ui (apurtell: rev 21e9074a90964ff374cf95897eae349ba21073a3) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2-addendum.patch, > HBASE-14976-v2.patch, HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14771) RpcServer#getRemoteAddress always returns null
[
https://issues.apache.org/jira/browse/HBASE-14771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103160#comment-15103160
]
Hudson commented on HBASE-14771:
FAILURE: Integrated in HBase-0.98-matrix #287 (See
[https://builds.apache.org/job/HBase-0.98-matrix/287/])
Amend HBASE-14771 RpcServer#getRemoteAddress always returns null (apurtell: rev
8189b0f255023c6d2e10565275979cea23f87e7d)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> RpcServer#getRemoteAddress always returns null
> --
>
> Key: HBASE-14771
> URL: https://issues.apache.org/jira/browse/HBASE-14771
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Affects Versions: 1.2.0
>Reporter: Abhishek Kumar
>Assignee: Abhishek Kumar
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14771-V2.patch, HBASE-14771-V1.patch,
> HBASE-14771-V2.patch, HBASE-14771-addendum-0.98.patch, HBASE-14771.patch
>
>
> RpcServer.getRemoteAddress always returns null, because Call object is
> getting initialized with null.This seems to be happening because of using
> RpcServer.getRemoteIp() in Call object constructor before RpcServer thread
> local 'CurCall' being set in CallRunner.run method:
> {noformat}
> // --- RpcServer.java ---
> protected void processRequest(byte[] buf) throws IOException,
> InterruptedException {
> .
> // Call object getting initialized here with address
> // obtained from RpcServer.getRemoteIp()
> Call call = new Call(id, this.service, md, header, param, cellScanner, this,
> responder,
> totalRequestSize, traceInfo, RpcServer.getRemoteIp());
> scheduler.dispatch(new CallRunner(RpcServer.this, call));
> }
> // getRemoteIp method gets address from threadlocal 'CurCall' which
> // gets set in CallRunner.run and calling it before this as in above case,
> will return null
> // --- CallRunner.java ---
> public void run() {
> .
> Pair resultPair = null;
> RpcServer.CurCall.set(call);
> ..
> }
> // Using 'this.addr' in place of getRemoteIp method in RpcServer.java seems
> to be fixing this issue
> Call call = new Call(id, this.service, md, header, param, cellScanner, this,
> responder,
> totalRequestSize, traceInfo, this.addr);
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15075) Allow region split request to carry identification information
[ https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15075: --- Attachment: HBASE-15075.v9.patch Patch v9 makes SimpleRegionNormalizer implement RegionStateListener. SimpleRegionNormalizer would receive notification about failed region splits and can use such information in normalization plan formation in subsequent iterations. > Allow region split request to carry identification information > -- > > Key: HBASE-15075 > URL: https://issues.apache.org/jira/browse/HBASE-15075 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0 > > Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt, > HBASE-15075.v2.patch, HBASE-15075.v3.patch, HBASE-15075.v4.patch, > HBASE-15075.v5.patch, HBASE-15075.v6.patch, HBASE-15075.v7.patch, > HBASE-15075.v8.patch, HBASE-15075.v9.patch > > > During the process of improving region normalization feature, I found that if > region split request triggered by the execution of SplitNormalizationPlan > fails, there is no way of knowing whether the failed split originated from > region normalization. > The association of particular split request with outcome of split would give > RegionNormalizer information so that it can make better normalization > decisions in the subsequent invocations. > One approach is to embed metadata, such as a UUID, in SplitRequest which gets > passed through RegionStateTransitionContext when > RegionServerServices#reportRegionStateTransition() is called. > This way, RegionStateListener can be notified with the metadata (id of the > requester). > See discussion on dev mailing list > http://search-hadoop.com/m/YGbbCXdkivihp2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14771) RpcServer#getRemoteAddress always returns null
[
https://issues.apache.org/jira/browse/HBASE-14771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103263#comment-15103263
]
Hudson commented on HBASE-14771:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1161 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1161/])
Amend HBASE-14771 RpcServer#getRemoteAddress always returns null (apurtell: rev
8189b0f255023c6d2e10565275979cea23f87e7d)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> RpcServer#getRemoteAddress always returns null
> --
>
> Key: HBASE-14771
> URL: https://issues.apache.org/jira/browse/HBASE-14771
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Affects Versions: 1.2.0
>Reporter: Abhishek Kumar
>Assignee: Abhishek Kumar
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14771-V2.patch, HBASE-14771-V1.patch,
> HBASE-14771-V2.patch, HBASE-14771-addendum-0.98.patch, HBASE-14771.patch
>
>
> RpcServer.getRemoteAddress always returns null, because Call object is
> getting initialized with null.This seems to be happening because of using
> RpcServer.getRemoteIp() in Call object constructor before RpcServer thread
> local 'CurCall' being set in CallRunner.run method:
> {noformat}
> // --- RpcServer.java ---
> protected void processRequest(byte[] buf) throws IOException,
> InterruptedException {
> .
> // Call object getting initialized here with address
> // obtained from RpcServer.getRemoteIp()
> Call call = new Call(id, this.service, md, header, param, cellScanner, this,
> responder,
> totalRequestSize, traceInfo, RpcServer.getRemoteIp());
> scheduler.dispatch(new CallRunner(RpcServer.this, call));
> }
> // getRemoteIp method gets address from threadlocal 'CurCall' which
> // gets set in CallRunner.run and calling it before this as in above case,
> will return null
> // --- CallRunner.java ---
> public void run() {
> .
> Pair resultPair = null;
> RpcServer.CurCall.set(call);
> ..
> }
> // Using 'this.addr' in place of getRemoteIp method in RpcServer.java seems
> to be fixing this issue
> Call call = new Call(id, this.service, md, header, param, cellScanner, this,
> responder,
> totalRequestSize, traceInfo, this.addr);
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14976) Add RPC call queues to the web ui
[ https://issues.apache.org/jira/browse/HBASE-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103262#comment-15103262 ] Hudson commented on HBASE-14976: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1161 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1161/]) Amend HBASE-14976 Add RPC call queues to the web ui (apurtell: rev 21e9074a90964ff374cf95897eae349ba21073a3) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon > Add RPC call queues to the web ui > - > > Key: HBASE-14976 > URL: https://issues.apache.org/jira/browse/HBASE-14976 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Elliott Clark >Assignee: Pallavi Adusumilli >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14976-v1.patch, HBASE-14976-v2-addendum.patch, > HBASE-14976-v2.patch, HBASE-14976-v2.patch, HBASE-14976.patch > > > The size of the call queue for the regionserver is a critical metric to see > if things are going too slowly. We should add the call queue size to the ui > under the queues tab. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14512) Cache UGI groups
[ https://issues.apache.org/jira/browse/HBASE-14512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103264#comment-15103264 ] Hudson commented on HBASE-14512: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1161 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1161/]) Amend HBASE-14512 Cache UGI groups (apurtell: rev d5f8300c082a75ce8edbbe08b66f077e7d663a4a) * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java > Cache UGI groups > > > Key: HBASE-14512 > URL: https://issues.apache.org/jira/browse/HBASE-14512 > Project: HBase > Issue Type: Bug > Components: Performance, security >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17 > > Attachments: HBASE-14512-addendum-0.98.patch, HBASE-14512-v1.patch, > HBASE-14512-v2.patch, HBASE-14512-v3.patch, HBASE-14512-v4.patch, > HBASE-14512.patch > > > Right now every call gets a new User object. > We should keep the same user for the life of a connection. We should also > cache the group names. However we can't cache the groups for forever as that > would mean groups don't get refreshed every 5 mins. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14969) Add throughput controller for flush
[
https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103274#comment-15103274
]
Yu Li commented on HBASE-14969:
---
Thanks for the review [~Apache9]
bq. upload the newest patch to review board
Just done.
bq. I suggest we also move compaction related controller classes into that
package
My only concern is that this will require user to modify their configuration if
use PressureAwareCompressionThroughputController. Please let me know your
thoughts here.
bq. And do we need two different {{NoLimitThroughputController}} for flush and
compaction?
Is it possible we add any separate logic in setup/start/finish method for flush
and compaction no-limit-controller in the future? If not, I agree we only
reserve NoLimitThroughputController and remove the other two.
> Add throughput controller for flush
> ---
>
> Key: HBASE-14969
> URL: https://issues.apache.org/jira/browse/HBASE-14969
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-14969.patch, HBASE-14969_v2.patch,
> HBASE-14969_v3.patch, HBASE-14969_v4.patch, HBASE-14969_v5.patch,
> HBASE-14969_v6.patch, load-nothrottling.log, load-throttling.log
>
>
> In HBASE-8329 we added a throughput controller for compaction, to avoid spike
> caused by huge IO pressure like network/disk overflow. However, even with
> this control on, we are still observing disk utils near 100%, and by analysis
> we think this is caused by flush, especially when we increase the setting of
> {{hbase.hstore.flusher.count}}
> In this JIRA, we propose to add throughput control feature for flush, as a
> supplement of HBASE-8329 to better control IO pressure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14962) TestSplitWalDataLoss fails on all branches
[
https://issues.apache.org/jira/browse/HBASE-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103428#comment-15103428
]
stack commented on HBASE-14962:
---
Will commit this small change soon (review meantime appreciated). The findbugs
failures are being worked on in another issue.
> TestSplitWalDataLoss fails on all branches
> --
>
> Key: HBASE-14962
> URL: https://issues.apache.org/jira/browse/HBASE-14962
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14962.patch, 14962v2.patch,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt
>
>
> With some regularity I am seeing:
> {code}
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: TestSplitWalDataLoss:dataloss: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1800(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1712)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15075) Allow region split request to carry identification information
[
https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103449#comment-15103449
]
Hadoop QA commented on HBASE-15075:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 29s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 53s
{color} | {color:red} hbase-server in master has 82 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 1s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 18s
{color} | {color:red} Patch generated 2 new checkstyle issues in hbase-server
(total was 574, now 555). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 12s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 39s
{color} | {color:green} hbase-protocol in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 5s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:red}-1{co
[jira] [Commented] (HBASE-15075) Allow region split request to carry identification information
[
https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103452#comment-15103452
]
Ted Yu commented on HBASE-15075:
>From
>https://builds.apache.org/job/PreCommit-HBASE-Build/159/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0.txt
> :
{code}
org.apache.hadoop.hbase.regionserver.TestHRegion.testBatchPut_whileNoRowLocksHeld(org.apache.hadoop.hbase.regionserver.TestHRegion)
Run 1: TestHRegion.testBatchPut_whileNoRowLocksHeld:1527 Metrics Counters
should be equal expected:<181> but was:<180>
Run 2: PASS
{code}
The flaky test was not related to the patch.
> Allow region split request to carry identification information
> --
>
> Key: HBASE-15075
> URL: https://issues.apache.org/jira/browse/HBASE-15075
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt,
> HBASE-15075.v2.patch, HBASE-15075.v3.patch, HBASE-15075.v4.patch,
> HBASE-15075.v5.patch, HBASE-15075.v6.patch, HBASE-15075.v7.patch,
> HBASE-15075.v8.patch, HBASE-15075.v9.patch
>
>
> During the process of improving region normalization feature, I found that if
> region split request triggered by the execution of SplitNormalizationPlan
> fails, there is no way of knowing whether the failed split originated from
> region normalization.
> The association of particular split request with outcome of split would give
> RegionNormalizer information so that it can make better normalization
> decisions in the subsequent invocations.
> One approach is to embed metadata, such as a UUID, in SplitRequest which gets
> passed through RegionStateTransitionContext when
> RegionServerServices#reportRegionStateTransition() is called.
> This way, RegionStateListener can be notified with the metadata (id of the
> requester).
> See discussion on dev mailing list
> http://search-hadoop.com/m/YGbbCXdkivihp2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15075) Allow region split request to carry identification information
[ https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103453#comment-15103453 ] Ted Yu commented on HBASE-15075: bq. JDK v1.8.0 Timed out junit tests org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters The above was not related to the patch either. TestReplicationAdminWithClusters passes locally. > Allow region split request to carry identification information > -- > > Key: HBASE-15075 > URL: https://issues.apache.org/jira/browse/HBASE-15075 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0 > > Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt, > HBASE-15075.v2.patch, HBASE-15075.v3.patch, HBASE-15075.v4.patch, > HBASE-15075.v5.patch, HBASE-15075.v6.patch, HBASE-15075.v7.patch, > HBASE-15075.v8.patch, HBASE-15075.v9.patch > > > During the process of improving region normalization feature, I found that if > region split request triggered by the execution of SplitNormalizationPlan > fails, there is no way of knowing whether the failed split originated from > region normalization. > The association of particular split request with outcome of split would give > RegionNormalizer information so that it can make better normalization > decisions in the subsequent invocations. > One approach is to embed metadata, such as a UUID, in SplitRequest which gets > passed through RegionStateTransitionContext when > RegionServerServices#reportRegionStateTransition() is called. > This way, RegionStateListener can be notified with the metadata (id of the > requester). > See discussion on dev mailing list > http://search-hadoop.com/m/YGbbCXdkivihp2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15075) Allow region split request to carry identification information
[ https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15075: --- Attachment: HBASE-15075.v10.patch Patch v10 addresses checkstyle warnings on import > Allow region split request to carry identification information > -- > > Key: HBASE-15075 > URL: https://issues.apache.org/jira/browse/HBASE-15075 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0 > > Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt, > HBASE-15075.v10.patch, HBASE-15075.v2.patch, HBASE-15075.v3.patch, > HBASE-15075.v4.patch, HBASE-15075.v5.patch, HBASE-15075.v6.patch, > HBASE-15075.v7.patch, HBASE-15075.v8.patch, HBASE-15075.v9.patch > > > During the process of improving region normalization feature, I found that if > region split request triggered by the execution of SplitNormalizationPlan > fails, there is no way of knowing whether the failed split originated from > region normalization. > The association of particular split request with outcome of split would give > RegionNormalizer information so that it can make better normalization > decisions in the subsequent invocations. > One approach is to embed metadata, such as a UUID, in SplitRequest which gets > passed through RegionStateTransitionContext when > RegionServerServices#reportRegionStateTransition() is called. > This way, RegionStateListener can be notified with the metadata (id of the > requester). > See discussion on dev mailing list > http://search-hadoop.com/m/YGbbCXdkivihp2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15075) Allow region split request to carry identification information
[ https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103493#comment-15103493 ] stack commented on HBASE-15075: --- Split that takes a UUID? Admin API is user-facing. What is user to make of "UUID"? Just split gets this attribute? Why don't all operations get a UUID... so can "identify" a request (whatever that means -- not clear from looking at patch). General Assignment region transition now takes a UUID but only used by split (confusing), and even more opaque, by 'normalization' only. Assignment is currently being redone. I've not seen mention of UUID per op to date. Did I miss something or will the new assignment take up this UUID and bake it in everywhere... > Allow region split request to carry identification information > -- > > Key: HBASE-15075 > URL: https://issues.apache.org/jira/browse/HBASE-15075 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0 > > Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt, > HBASE-15075.v10.patch, HBASE-15075.v2.patch, HBASE-15075.v3.patch, > HBASE-15075.v4.patch, HBASE-15075.v5.patch, HBASE-15075.v6.patch, > HBASE-15075.v7.patch, HBASE-15075.v8.patch, HBASE-15075.v9.patch > > > During the process of improving region normalization feature, I found that if > region split request triggered by the execution of SplitNormalizationPlan > fails, there is no way of knowing whether the failed split originated from > region normalization. > The association of particular split request with outcome of split would give > RegionNormalizer information so that it can make better normalization > decisions in the subsequent invocations. > One approach is to embed metadata, such as a UUID, in SplitRequest which gets > passed through RegionStateTransitionContext when > RegionServerServices#reportRegionStateTransition() is called. > This way, RegionStateListener can be notified with the metadata (id of the > requester). > See discussion on dev mailing list > http://search-hadoop.com/m/YGbbCXdkivihp2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15119) Include git SHA in check_compatibility reports
[ https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-15119: - Attachment: HBASE-15119.v00.patch This appears to do the trick. [~busbey], [~dimaspivak]? > Include git SHA in check_compatibility reports > -- > > Key: HBASE-15119 > URL: https://issues.apache.org/jira/browse/HBASE-15119 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15119.v00.patch > > > Since some refs change over time (ie, branches), it would be nice to include > git shas in the version info included in check compatibility reports. It'll > also help interested parties to be sure of what they're looking at. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15119) Include git SHA in check_compatibility reports
[ https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-15119: - Status: Patch Available (was: Open) > Include git SHA in check_compatibility reports > -- > > Key: HBASE-15119 > URL: https://issues.apache.org/jira/browse/HBASE-15119 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15119.v00.patch > > > Since some refs change over time (ie, branches), it would be nice to include > git shas in the version info included in check compatibility reports. It'll > also help interested parties to be sure of what they're looking at. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14962) TestSplitWalDataLoss fails on all branches
[
https://issues.apache.org/jira/browse/HBASE-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14962:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to branch 1.2+. Small change. Ran on the hardware where it used fail 100
times successfully.
> TestSplitWalDataLoss fails on all branches
> --
>
> Key: HBASE-14962
> URL: https://issues.apache.org/jira/browse/HBASE-14962
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14962.patch, 14962v2.patch,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt
>
>
> With some regularity I am seeing:
> {code}
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: TestSplitWalDataLoss:dataloss: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1800(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1712)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15091) Forward-port to 1.2 HBASE-15031 "Fix merge of MVCC and SequenceID performance regression in branch-1.0"
[ https://issues.apache.org/jira/browse/HBASE-15091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15091: -- Attachment: 15091v6.branch-1.2.patch The 'bad commit' must still have been in place. Retrying v6 of the patch. > Forward-port to 1.2 HBASE-15031 "Fix merge of MVCC and SequenceID performance > regression in branch-1.0" > --- > > Key: HBASE-15091 > URL: https://issues.apache.org/jira/browse/HBASE-15091 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Priority: Blocker > Fix For: 1.2.0 > > Attachments: 15091v2.branch-1.2.patch, 15091v3.branch-1.2.patch, > 15091v4.branch-1.2.patch, 15091v5.branch-1.2.patch, 15091v6.branch-1.2.patch, > 15091v6.branch-1.patch, HBASE-15091-branch-1.2.patch, > HBASE-15091-branch-1.2_v1.patch, HBASE-15091.v1.branch-1.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15119) Include git SHA in check_compatibility reports
[
https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103499#comment-15103499
]
Dima Spivak commented on HBASE-15119:
-
Just wondering, do we really need
{noformat}
+JAVA_ACC_COMMAND+=(-v1 ${COMMIT[1]}${SHA[1]+"/${SHA[1]}"})
+JAVA_ACC_COMMAND+=(-v2 ${COMMIT[2]}${SHA[2]+"/${SHA[2]}"})
{noformat}
to be set? Seems to me like just having ${COMMIT[1]}/${SHA[1]} would have the
same effect here (minus an extra slash), but I guess I can't see why revparse
would fail. Otherwise, LGTM.
> Include git SHA in check_compatibility reports
> --
>
> Key: HBASE-15119
> URL: https://issues.apache.org/jira/browse/HBASE-15119
> Project: HBase
> Issue Type: Improvement
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15119.v00.patch
>
>
> Since some refs change over time (ie, branches), it would be nice to include
> git shas in the version info included in check compatibility reports. It'll
> also help interested parties to be sure of what they're looking at.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15119) Include git SHA in check_compatibility reports
[
https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103501#comment-15103501
]
Hadoop QA commented on HBASE-15119:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 20s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 42s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 4s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 24s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 43s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 5s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 27s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 46s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 12m 7s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 31s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12782740/HBASE-15119.v00.patch
|
| JIRA Issue | HBASE-15119 |
| Optional Tests | asflicense shellcheck |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 387eaff |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| modules | C: . U: . |
| Max memory used | 160MB |
| Powered by | Apache Yetus 0.1.0 http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/163/console |
This message was automatically generated.
> Include git SHA in check_compatibility reports
> --
>
> Key: HBASE-15119
> URL: https://issues.apache.org/jira/browse/HBASE-15119
> Project: HBase
> Issue Type: Improvement
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15119.v00.patch
>
>
> Since some refs change over time (ie, branches), it would be nice to include
> git shas in the version info included in check compatibility reports. It'll
> also help interested parties to be sure of what they're looking at.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15119) Include git SHA in check_compatibility reports
[
https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103502#comment-15103502
]
Dima Spivak commented on HBASE-15119:
-
Oy. Damn you, JIRA. I meant that having {code}${COMMIT[1]}/${SHA[1]}{code}would
have the same effect (albeit with a hanging slash at the end if the variable
were missing).
> Include git SHA in check_compatibility reports
> --
>
> Key: HBASE-15119
> URL: https://issues.apache.org/jira/browse/HBASE-15119
> Project: HBase
> Issue Type: Improvement
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15119.v00.patch
>
>
> Since some refs change over time (ie, branches), it would be nice to include
> git shas in the version info included in check compatibility reports. It'll
> also help interested parties to be sure of what they're looking at.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15115) Fix findbugs complaints in hbase-client
[ https://issues.apache.org/jira/browse/HBASE-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15115: -- Resolution: Fixed Fix Version/s: 1.3.0 1.2.0 2.0.0 Status: Resolved (was: Patch Available) Looks like no more client findbugs running a null patch. Backported the findbugs changes to branch-1 and branch-1.2. > Fix findbugs complaints in hbase-client > --- > > Key: HBASE-15115 > URL: https://issues.apache.org/jira/browse/HBASE-15115 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 15115.patch, none_fix.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15118) Fix findbugs complaint in hbase-server
[ https://issues.apache.org/jira/browse/HBASE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15118: -- Status: Patch Available (was: Open) > Fix findbugs complaint in hbase-server > -- > > Key: HBASE-15118 > URL: https://issues.apache.org/jira/browse/HBASE-15118 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack > Attachments: 15118.patch, 15118v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13103) [ergonomics] add region size balancing as a feature of master
[
https://issues.apache.org/jira/browse/HBASE-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103516#comment-15103516
]
stack commented on HBASE-13103:
---
[~mantonov] The release note is no correct (or circumstance has changed since
you wrote it?) It says this feature is off but it is on by default in 1.2,
right sir?
> [ergonomics] add region size balancing as a feature of master
> -
>
> Key: HBASE-13103
> URL: https://issues.apache.org/jira/browse/HBASE-13103
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Usability
>Reporter: Nick Dimiduk
>Assignee: Mikhail Antonov
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-13103-branch-1.v3.patch, HBASE-13103-v0.patch,
> HBASE-13103-v1.patch, HBASE-13103-v2.patch, HBASE-13103-v3.patch
>
>
> Often enough, folks miss-judge split points or otherwise end up with a
> suboptimal number of regions. We should have an automated, reliable way to
> "reshape" or "balance" a table's region boundaries. This would be for tables
> that contain existing data. This might look like:
> {noformat}
> Admin#reshapeTable(TableName, int numSplits);
> {noformat}
> or from the shell:
> {noformat}
> > reshape TABLE, numSplits
> {noformat}
> Better still would be to have a maintenance process, similar to the existing
> Balancer that runs AssignmentManager on an interval, to run the above
> "reshape" operation on an interval. That way, the cluster will automatically
> self-correct toward a desirable state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14367) Add normalization support to shell
[
https://issues.apache.org/jira/browse/HBASE-14367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14367:
--
Release Note:
This patch adds shell support for region normalizer (see HBASE-13103).
3 commands have been added to hbase shell 'tools' command group (modeled on how
the balancer works):
- 'normalizer_enabled' checks whether region normalizer is turned on
- 'normalizer_switch' allows user to turn normalizer on and off
- 'normalize' runs region normalizer if it's turned on.
Also 'alter' command has been extended to allow user to enable/disable region
normalization per table (disabled by default). Use it as
alter 'testtable', {NORMALIZATION_MODE => 'true'}
Here is the help for the normalize command:
{code}
hbase(main):008:0> help 'normalize'
Trigger region normalizer for all tables which have NORMALIZATION_MODE flag
set. Returns true
if normalizer ran successfully, false otherwise. Note that this command has no
effect
if region normalizer is disabled (make sure it's turned on using
'normalizer_switch' command).
Examples:
hbase> normalize
{code}
was:
This patch adds shell support for region normalizer.
3 commands have been added to hbase shell 'tools' command group:
- 'normalizer_enabled' checks whether region normalizer is turned on
- 'normalizer_switch' allows user to turn normalizer on and off
- 'normalize' runs region normalizer if it's turned on.
Also 'ater' command has been extended to allow user to enable/disable region
normalization per table (disabled by default). Use it as
alter 'testtable', {NORMALIZATION_ENABLED => 'true'}
> Add normalization support to shell
> --
>
> Key: HBASE-14367
> URL: https://issues.apache.org/jira/browse/HBASE-14367
> Project: HBase
> Issue Type: Bug
> Components: Balancer, shell
>Affects Versions: 1.1.2
>Reporter: Lars George
>Assignee: Mikhail Antonov
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14367-branch-1.2.v1.patch,
> HBASE-14367-branch-1.2.v2.patch, HBASE-14367-branch-1.2.v3.patch,
> HBASE-14367-branch-1.v1.patch, HBASE-14367-v1.patch, HBASE-14367.patch
>
>
> https://issues.apache.org/jira/browse/HBASE-13103 adds support for setting a
> normalization flag per {{HTableDescriptor}}, along with the server side chore
> to do the work.
> What is lacking is to easily set this from the shell, right now you need to
> use the Java API to modify the descriptor. This issue is to add the flag as a
> known attribute key and/or other means to toggle this per table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13103) [ergonomics] add region size balancing as a feature of master
[
https://issues.apache.org/jira/browse/HBASE-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103526#comment-15103526
]
stack commented on HBASE-13103:
---
Yeah, and add a note on what the 'normalize' process is sir. Thanks boss.
> [ergonomics] add region size balancing as a feature of master
> -
>
> Key: HBASE-13103
> URL: https://issues.apache.org/jira/browse/HBASE-13103
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Usability
>Reporter: Nick Dimiduk
>Assignee: Mikhail Antonov
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-13103-branch-1.v3.patch, HBASE-13103-v0.patch,
> HBASE-13103-v1.patch, HBASE-13103-v2.patch, HBASE-13103-v3.patch
>
>
> Often enough, folks miss-judge split points or otherwise end up with a
> suboptimal number of regions. We should have an automated, reliable way to
> "reshape" or "balance" a table's region boundaries. This would be for tables
> that contain existing data. This might look like:
> {noformat}
> Admin#reshapeTable(TableName, int numSplits);
> {noformat}
> or from the shell:
> {noformat}
> > reshape TABLE, numSplits
> {noformat}
> Better still would be to have a maintenance process, similar to the existing
> Balancer that runs AssignmentManager on an interval, to run the above
> "reshape" operation on an interval. That way, the cluster will automatically
> self-correct toward a desirable state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14367) Add normalization support to shell
[
https://issues.apache.org/jira/browse/HBASE-14367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103528#comment-15103528
]
stack commented on HBASE-14367:
---
The attribute on the table's name change from NORMALIZATION_ENABLED to
NORMALIZATION_MODE over in HBASE-15073... I updated the release note here to
try and save on confusion if folks saw NORMALIZATION_ENABLED
> Add normalization support to shell
> --
>
> Key: HBASE-14367
> URL: https://issues.apache.org/jira/browse/HBASE-14367
> Project: HBase
> Issue Type: Bug
> Components: Balancer, shell
>Affects Versions: 1.1.2
>Reporter: Lars George
>Assignee: Mikhail Antonov
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14367-branch-1.2.v1.patch,
> HBASE-14367-branch-1.2.v2.patch, HBASE-14367-branch-1.2.v3.patch,
> HBASE-14367-branch-1.v1.patch, HBASE-14367-v1.patch, HBASE-14367.patch
>
>
> https://issues.apache.org/jira/browse/HBASE-13103 adds support for setting a
> normalization flag per {{HTableDescriptor}}, along with the server side chore
> to do the work.
> What is lacking is to easily set this from the shell, right now you need to
> use the Java API to modify the descriptor. This issue is to add the flag as a
> known attribute key and/or other means to toggle this per table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103530#comment-15103530
]
stack commented on HBASE-15073:
---
I'm trying to trace the new 'normalization' feature.
What is this about?
bq. However, for certain use case(s), it would be desirable to perform only one
type of action.
A statement about 'cases' but no cases are provided.
Why do we need finer grained control? The point is that this is an ergonomic
feature that lets the machine do it for you... if need admin intervention, then
the feature is not working right... lets fix that. This issue seems to mangle
the intent of 'normalization'.
Looking at patch I see a define: DESIRED_NORMALIZATION_TYPES What does DESIRED
mean? This is a datastore, not a beach at sunset...
Users are going to see 'MS', 'M', and 'S' and are supposed to get split, merge,
etc. from this? That's not nice on our user.
I'm -1 on this thing till its explained why. I think should revert from 1.2.
Reopening.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15118) Fix findbugs complaint in hbase-server
[
https://issues.apache.org/jira/browse/HBASE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103539#comment-15103539
]
Hadoop QA commented on HBASE-15118:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 51s
{color} | {color:red} hbase-server in master has 82 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 24s
{color} | {color:red} Patch generated 6 new checkstyle issues in hbase-server
(total was 1534, now 1533). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 22s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 44s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 4s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 26s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 48s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 12s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 33s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 54s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 12m 18s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {colo
[jira] [Commented] (HBASE-15075) Allow region split request to carry identification information
[ https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103544#comment-15103544 ] stack commented on HBASE-15075: --- Lets hold on this development. It is predicated on 'During the process of improving region normalization feature...'. My reading of the work 'improving' region normalization is that normalization is being mangled, not improved That would undo the predicate for this work. > Allow region split request to carry identification information > -- > > Key: HBASE-15075 > URL: https://issues.apache.org/jira/browse/HBASE-15075 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0 > > Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt, > HBASE-15075.v10.patch, HBASE-15075.v2.patch, HBASE-15075.v3.patch, > HBASE-15075.v4.patch, HBASE-15075.v5.patch, HBASE-15075.v6.patch, > HBASE-15075.v7.patch, HBASE-15075.v8.patch, HBASE-15075.v9.patch > > > During the process of improving region normalization feature, I found that if > region split request triggered by the execution of SplitNormalizationPlan > fails, there is no way of knowing whether the failed split originated from > region normalization. > The association of particular split request with outcome of split would give > RegionNormalizer information so that it can make better normalization > decisions in the subsequent invocations. > One approach is to embed metadata, such as a UUID, in SplitRequest which gets > passed through RegionStateTransitionContext when > RegionServerServices#reportRegionStateTransition() is called. > This way, RegionStateListener can be notified with the metadata (id of the > requester). > See discussion on dev mailing list > http://search-hadoop.com/m/YGbbCXdkivihp2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103546#comment-15103546
]
Ted Yu commented on HBASE-15073:
bq. certain use case(s)
One use case is for time series data where FIFO compaction policy is enforced.
There would be regions whose data expires gradually and gets cleaned by the
compaction policy, leaving empty regions behind.
Region normalizer would issue region merge requests for the empty regions so
that the number of regions for the table is under control.
In the above scenario, region split request during normalization doesn't play
as important a role as that for region merge. Therefore user has the choice of
turning off region split (during normalization).
w.r.t. DESIRED_NORMALIZATION_TYPES, I don't see it in
hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
(master branch, branch-1 or branch-1.2)
It has been dropped since it is not referenced.
bq. Users are going to see 'MS', 'M', and 'S' and are supposed to get split,
merge, etc. from this?
Yes.
I am open to suggestion on better encoding for these two operations.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15075) Allow region split request to carry identification information
[
https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103548#comment-15103548
]
Hadoop QA commented on HBASE-15075:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 59s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 49s
{color} | {color:red} hbase-server in master has 82 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 37s
{color} | {color:green} hbase-protocol in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 4s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 16s
[jira] [Commented] (HBASE-13103) [ergonomics] add region size balancing as a feature of master
[
https://issues.apache.org/jira/browse/HBASE-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103549#comment-15103549
]
stack commented on HBASE-13103:
---
nvm [~mantonov] Lets just work on a bit of doc for this new feature instead.
Release was 'correct' as of writing.
> [ergonomics] add region size balancing as a feature of master
> -
>
> Key: HBASE-13103
> URL: https://issues.apache.org/jira/browse/HBASE-13103
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Usability
>Reporter: Nick Dimiduk
>Assignee: Mikhail Antonov
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-13103-branch-1.v3.patch, HBASE-13103-v0.patch,
> HBASE-13103-v1.patch, HBASE-13103-v2.patch, HBASE-13103-v3.patch
>
>
> Often enough, folks miss-judge split points or otherwise end up with a
> suboptimal number of regions. We should have an automated, reliable way to
> "reshape" or "balance" a table's region boundaries. This would be for tables
> that contain existing data. This might look like:
> {noformat}
> Admin#reshapeTable(TableName, int numSplits);
> {noformat}
> or from the shell:
> {noformat}
> > reshape TABLE, numSplits
> {noformat}
> Better still would be to have a maintenance process, similar to the existing
> Balancer that runs AssignmentManager on an interval, to run the above
> "reshape" operation on an interval. That way, the cluster will automatically
> self-correct toward a desirable state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15115) Fix findbugs complaints in hbase-client
[ https://issues.apache.org/jira/browse/HBASE-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103551#comment-15103551 ] Hudson commented on HBASE-15115: SUCCESS: Integrated in HBase-1.3 #498 (See [https://builds.apache.org/job/HBase-1.3/498/]) HBASE-15115 Fix findbugs complaints in hbase-client (stack: rev d9fd87d5a8b8593ec2ec275b0395bd872d19cf04) * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/LongComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingCallerInterceptorFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/MultiRowRangeFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedException.java > Fix findbugs complaints in hbase-client > --- > > Key: HBASE-15115 > URL: https://issues.apache.org/jira/browse/HBASE-15115 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 15115.patch, none_fix.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14962) TestSplitWalDataLoss fails on all branches
[
https://issues.apache.org/jira/browse/HBASE-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103552#comment-15103552
]
Hudson commented on HBASE-14962:
SUCCESS: Integrated in HBase-1.3 #498 (See
[https://builds.apache.org/job/HBase-1.3/498/])
HBASE-14962 TestSplitWalDataLoss fails on all branches (stack: rev
032f5574b01ea1eb87453be1231570b70a34e15d)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitWalDataLoss.java
> TestSplitWalDataLoss fails on all branches
> --
>
> Key: HBASE-14962
> URL: https://issues.apache.org/jira/browse/HBASE-14962
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14962.patch, 14962v2.patch,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt
>
>
> With some regularity I am seeing:
> {code}
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: TestSplitWalDataLoss:dataloss: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1800(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1712)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15115) Fix findbugs complaints in hbase-client
[ https://issues.apache.org/jira/browse/HBASE-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103554#comment-15103554 ] Hudson commented on HBASE-15115: SUCCESS: Integrated in HBase-1.3-IT #441 (See [https://builds.apache.org/job/HBase-1.3-IT/441/]) HBASE-15115 Fix findbugs complaints in hbase-client (stack: rev d9fd87d5a8b8593ec2ec275b0395bd872d19cf04) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/LongComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingCallerInterceptorFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/MultiRowRangeFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java > Fix findbugs complaints in hbase-client > --- > > Key: HBASE-15115 > URL: https://issues.apache.org/jira/browse/HBASE-15115 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 15115.patch, none_fix.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14962) TestSplitWalDataLoss fails on all branches
[
https://issues.apache.org/jira/browse/HBASE-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103555#comment-15103555
]
Hudson commented on HBASE-14962:
SUCCESS: Integrated in HBase-1.3-IT #441 (See
[https://builds.apache.org/job/HBase-1.3-IT/441/])
HBASE-14962 TestSplitWalDataLoss fails on all branches (stack: rev
032f5574b01ea1eb87453be1231570b70a34e15d)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitWalDataLoss.java
> TestSplitWalDataLoss fails on all branches
> --
>
> Key: HBASE-14962
> URL: https://issues.apache.org/jira/browse/HBASE-14962
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14962.patch, 14962v2.patch,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt
>
>
> With some regularity I am seeing:
> {code}
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: TestSplitWalDataLoss:dataloss: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1800(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1712)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15124) Document the new 'normalization' feature in refguid
stack created HBASE-15124: - Summary: Document the new 'normalization' feature in refguid Key: HBASE-15124 URL: https://issues.apache.org/jira/browse/HBASE-15124 Project: HBase Issue Type: Task Components: documentation Reporter: stack Priority: Critical Fix For: 1.2.0 A nice new feature is coming in to 1.2.0, normalization. A small bit of doc on it in refguide would help. Should define what normalization is. Should say a sentence or two on how it works and when. Throw in the output of shell commands. A paragraph or so. I can help. Marking critical against 1.2.0. Not a blocker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15091) Forward-port to 1.2 HBASE-15031 "Fix merge of MVCC and SequenceID performance regression in branch-1.0"
[
https://issues.apache.org/jira/browse/HBASE-15091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103558#comment-15103558
]
Hadoop QA commented on HBASE-15091:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 12 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
18s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s {color} | {color:green} branch-1.2 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 55s
{color} | {color:red} hbase-client in branch-1.2 has 17 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 41s
{color} | {color:red} hbase-common in branch-1.2 has 1 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 46s
{color} | {color:red} hbase-server in branch-1.2 has 84 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 16s
{color} | {color:red} Patch generated 1 new checkstyle issues in hbase-server
(total was 78, now 77). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 30s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 53s
{color} | {color:red} hbase-common introduced 1 new FindBugs issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 31s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 41s
{color} | {color:green} hbase-common in the patch passed with JDK v1.8.0.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 25s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 38s
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 53s
{color} | {color:green} hbase-common in the pa
[jira] [Commented] (HBASE-14962) TestSplitWalDataLoss fails on all branches
[
https://issues.apache.org/jira/browse/HBASE-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103566#comment-15103566
]
Hudson commented on HBASE-14962:
SUCCESS: Integrated in HBase-1.2-IT #399 (See
[https://builds.apache.org/job/HBase-1.2-IT/399/])
HBASE-14962 TestSplitWalDataLoss fails on all branches (stack: rev
36d1adb6065a20208ea2c43130abd021856011dd)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitWalDataLoss.java
> TestSplitWalDataLoss fails on all branches
> --
>
> Key: HBASE-14962
> URL: https://issues.apache.org/jira/browse/HBASE-14962
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14962.patch, 14962v2.patch,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt
>
>
> With some regularity I am seeing:
> {code}
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: TestSplitWalDataLoss:dataloss: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1800(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1712)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15115) Fix findbugs complaints in hbase-client
[ https://issues.apache.org/jira/browse/HBASE-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103565#comment-15103565 ] Hudson commented on HBASE-15115: SUCCESS: Integrated in HBase-1.2-IT #399 (See [https://builds.apache.org/job/HBase-1.2-IT/399/]) HBASE-15115 Fix findbugs complaints in hbase-client (stack: rev 5a280c9fd63376aec8184ee74ede1d7de3e0d56a) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/LongComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/MultiRowRangeFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingCallerInterceptorFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedException.java > Fix findbugs complaints in hbase-client > --- > > Key: HBASE-15115 > URL: https://issues.apache.org/jira/browse/HBASE-15115 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 15115.patch, none_fix.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14962) TestSplitWalDataLoss fails on all branches
[
https://issues.apache.org/jira/browse/HBASE-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103567#comment-15103567
]
Hudson commented on HBASE-14962:
FAILURE: Integrated in HBase-Trunk_matrix #639 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/639/])
HBASE-14962 TestSplitWalDataLoss fails on all branches (stack: rev
47fc696bc61e10386cf35644d95d2f45246d3269)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitWalDataLoss.java
> TestSplitWalDataLoss fails on all branches
> --
>
> Key: HBASE-14962
> URL: https://issues.apache.org/jira/browse/HBASE-14962
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14962.patch, 14962v2.patch,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt
>
>
> With some regularity I am seeing:
> {code}
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: TestSplitWalDataLoss:dataloss: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1800(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1712)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15098) Normalizer switch in configuration is not used
[
https://issues.apache.org/jira/browse/HBASE-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103577#comment-15103577
]
stack commented on HBASE-15098:
---
bq. We should implement a proper if statement that checks this value and
properly disables the feature cluster wide if not wanted.
A kill switch @lars george? I think it good that the 'normalizing' feature is
'on' by default but only has an effect if you ask for it in a table attribute.
In that case, the attached patch would make sense; it cleans up a config left
over from when the though was you'd opt in. But if you want a kill switch, we
should bring back the config and make code take notice (Myself, I don't think
we need a kill switch). Thanks Lars.
> Normalizer switch in configuration is not used
> --
>
> Key: HBASE-15098
> URL: https://issues.apache.org/jira/browse/HBASE-15098
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.2.0
>Reporter: Lars George
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-15098.v1.patch
>
>
> The newly added global switch to enable the new normalizer functionality is
> never used apparently, meaning it is always on. The {{hbase-default.xml}} has
> this:
> {noformat}
>
> hbase.normalizer.enabled
> false
> If set to true, Master will try to keep region size
> within each table approximately the same.
>
> {noformat}
> But only a test class uses it to set the switch to "true". We should
> implement a proper {{if}} statement that checks this value and properly
> disables the feature cluster wide if not wanted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15124) Document the new 'normalization' feature in refguid
[ https://issues.apache.org/jira/browse/HBASE-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103588#comment-15103588 ] stack commented on HBASE-15124: --- Doc should say this a new feature. Experimental. > Document the new 'normalization' feature in refguid > --- > > Key: HBASE-15124 > URL: https://issues.apache.org/jira/browse/HBASE-15124 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack >Priority: Critical > Fix For: 1.2.0 > > > A nice new feature is coming in to 1.2.0, normalization. A small bit of doc > on it in refguide would help. > Should define what normalization is. > Should say a sentence or two on how it works and when. > Throw in the output of shell commands. > A paragraph or so. I can help. > Marking critical against 1.2.0. Not a blocker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103608#comment-15103608
]
stack commented on HBASE-15073:
---
bq. In the above scenario, region split request during normalization doesn't
play as important a role as that for region merge.
Sure.
bq. Therefore user has the choice of turning off region split (during
normalization).
I don't follow. Would a split even be issued? The merge is keeping table
regions tidy. And then why would I want the 'choice' of turning off splits?
This is all an intellectual exercise, right? Any evidence from a running
cluster to backup need for finer grainer control?
Any other use cases other than this "If timeseries and if FIFO compaction and
if the user wants to turn off splits."?
Users don't want choice. There is too much 'choice' in hbase already. They just
want hbase to make the right choice.
bq. I am open to suggestion on better encoding for these two operations.
I suggest just purging this pursuit of 'finer grained control' over
normalization and instead try and go the other direction where you turn
normalization on and it just does the right thing autonomously... then no need
of this 'MS' or 'S' or 'M'... stuff.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15082) Fix merge of MVCC and SequenceID performance regression
[ https://issues.apache.org/jira/browse/HBASE-15082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15082: -- Attachment: 15082v14.patch Lets see how this patch does now. > Fix merge of MVCC and SequenceID performance regression > --- > > Key: HBASE-15082 > URL: https://issues.apache.org/jira/browse/HBASE-15082 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: 15082.patch, 15082v10.patch, 15082v12.patch, > 15082v13.patch, 15082v14.patch, 15082v2.patch, 15082v2.txt, 15082v3.txt, > 15082v4.patch, 15082v5.patch, 15082v6.patch, 15082v7.patch, 15082v8.patch > > > This is general fix for increments (appends, checkAnd*) perf-regression > identified in the parent issue. HBASE-15031 has a narrow fix for branch-1.1 > and branch-1.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15115) Fix findbugs complaints in hbase-client
[ https://issues.apache.org/jira/browse/HBASE-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103612#comment-15103612 ] Hudson commented on HBASE-15115: FAILURE: Integrated in HBase-1.2 #508 (See [https://builds.apache.org/job/HBase-1.2/508/]) HBASE-15115 Fix findbugs complaints in hbase-client (stack: rev 5a280c9fd63376aec8184ee74ede1d7de3e0d56a) * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetryingCallerInterceptorFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedException.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/LongComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/MultiRowRangeFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java > Fix findbugs complaints in hbase-client > --- > > Key: HBASE-15115 > URL: https://issues.apache.org/jira/browse/HBASE-15115 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 15115.patch, none_fix.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14962) TestSplitWalDataLoss fails on all branches
[
https://issues.apache.org/jira/browse/HBASE-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103613#comment-15103613
]
Hudson commented on HBASE-14962:
FAILURE: Integrated in HBase-1.2 #508 (See
[https://builds.apache.org/job/HBase-1.2/508/])
HBASE-14962 TestSplitWalDataLoss fails on all branches (stack: rev
36d1adb6065a20208ea2c43130abd021856011dd)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitWalDataLoss.java
> TestSplitWalDataLoss fails on all branches
> --
>
> Key: HBASE-14962
> URL: https://issues.apache.org/jira/browse/HBASE-14962
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14962.patch, 14962v2.patch,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt,
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss-output.txt
>
>
> With some regularity I am seeing:
> {code}
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: TestSplitWalDataLoss:dataloss: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1800(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1712)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:121)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15122) Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
[
https://issues.apache.org/jira/browse/HBASE-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15122:
--
Attachment: HBASE-15122.patch
make a patch. :)
> Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
> ---
>
> Key: HBASE-15122
> URL: https://issues.apache.org/jira/browse/HBASE-15122
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Attachments: HBASE-15122.patch
>
>
> In our JMXJsonServlet we are doing this:
> jsonpcb = request.getParameter(CALLBACK_PARAM);
> if (jsonpcb != null) {
> response.setContentType("application/javascript; charset=utf8");
> writer.write(jsonpcb + "(");
> ...
> Findbugs complains rightly. There are other instances in our servlets and
> then there are the pages generated by jamon excluded from findbugs checking
> (and findbugs volunteers that it is dumb in this regard finding only the most
> egregious of violations).
> We have no sanitizing tooling in hbase that I know of (correct me if I am
> wrong). I started to pull on this thread and it runs deep. Our Jamon
> templating (last updated in 2013 and before that, in 2011) engine doesn't
> seem to have sanitizing means either and there seems to be outstanding XSS
> complaint against jamon that goes unaddressed.
> Could pull in something like
> https://www.owasp.org/index.php/OWASP_Java_Encoder_Project and run all
> emissions via it or get a templating engine that has sanitizing built in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15122) Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
[
https://issues.apache.org/jira/browse/HBASE-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15122:
--
Status: Patch Available (was: Open)
> Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
> ---
>
> Key: HBASE-15122
> URL: https://issues.apache.org/jira/browse/HBASE-15122
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Attachments: HBASE-15122.patch
>
>
> In our JMXJsonServlet we are doing this:
> jsonpcb = request.getParameter(CALLBACK_PARAM);
> if (jsonpcb != null) {
> response.setContentType("application/javascript; charset=utf8");
> writer.write(jsonpcb + "(");
> ...
> Findbugs complains rightly. There are other instances in our servlets and
> then there are the pages generated by jamon excluded from findbugs checking
> (and findbugs volunteers that it is dumb in this regard finding only the most
> egregious of violations).
> We have no sanitizing tooling in hbase that I know of (correct me if I am
> wrong). I started to pull on this thread and it runs deep. Our Jamon
> templating (last updated in 2013 and before that, in 2011) engine doesn't
> seem to have sanitizing means either and there seems to be outstanding XSS
> complaint against jamon that goes unaddressed.
> Could pull in something like
> https://www.owasp.org/index.php/OWASP_Java_Encoder_Project and run all
> emissions via it or get a templating engine that has sanitizing built in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15122) Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
[
https://issues.apache.org/jira/browse/HBASE-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103624#comment-15103624
]
stack commented on HBASE-15122:
---
Excellent. I like the way you add tooling so we can address this stuff going
forward.
[~apurtell] (or [~busbey]/[~enis]) Any input? We are probably susceptible in a
million places. This is the one place found by FB. It says that if it is
finding XSS, then very likely there is much more to be found (other tooling
that looks for XSS will turn up instances. Our templating system is dumb (and
old). Has no facility for sanitizing against XSS. Failing that, this lib
introduced in this patch gives us ammo going forward.
> Servlets generate XSS_REQUEST_PARAMETER_TO_SERVLET_WRITER findbugs warnings
> ---
>
> Key: HBASE-15122
> URL: https://issues.apache.org/jira/browse/HBASE-15122
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Attachments: HBASE-15122.patch
>
>
> In our JMXJsonServlet we are doing this:
> jsonpcb = request.getParameter(CALLBACK_PARAM);
> if (jsonpcb != null) {
> response.setContentType("application/javascript; charset=utf8");
> writer.write(jsonpcb + "(");
> ...
> Findbugs complains rightly. There are other instances in our servlets and
> then there are the pages generated by jamon excluded from findbugs checking
> (and findbugs volunteers that it is dumb in this regard finding only the most
> egregious of violations).
> We have no sanitizing tooling in hbase that I know of (correct me if I am
> wrong). I started to pull on this thread and it runs deep. Our Jamon
> templating (last updated in 2013 and before that, in 2011) engine doesn't
> seem to have sanitizing means either and there seems to be outstanding XSS
> complaint against jamon that goes unaddressed.
> Could pull in something like
> https://www.owasp.org/index.php/OWASP_Java_Encoder_Project and run all
> emissions via it or get a templating engine that has sanitizing built in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15118) Fix findbugs complaint in hbase-server
[ https://issues.apache.org/jira/browse/HBASE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15118: -- Attachment: 15118v3.patch Fix checkstyle and the unit test failure was legit (an assert I'd added to satisfy FindBugs check was throwing... bad assert) > Fix findbugs complaint in hbase-server > -- > > Key: HBASE-15118 > URL: https://issues.apache.org/jira/browse/HBASE-15118 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack > Attachments: 15118.patch, 15118v2.patch, 15118v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15082) Fix merge of MVCC and SequenceID performance regression
[
https://issues.apache.org/jira/browse/HBASE-15082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103630#comment-15103630
]
Hadoop QA commented on HBASE-15082:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 42s
{color} | {color:red} hbase-common in master has 1 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 49s
{color} | {color:red} hbase-server in master has 82 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 36s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 13s
{color} | {color:red} hbase-examples in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 22s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 19s
{color} | {color:red} hbase-examples in the patch failed with JDK v1.8.0.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 32s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0. {color}
|
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 19s {color}
| {color:red} hbase-examples in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 32s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 13s
{color} | {color:red} hbase-examples in the patch failed with JDK v1.7.0_79.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 21s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_79.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 13s {color}
| {color:red} hbase-examples in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 21s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 9s
{color} | {color:red} Patch generated 3 new checkstyle issues in hbase-common
(total was 96, now 99). {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 22s
{color} | {color:red} Patch generated 86 new checkstyle issues in hbase-server
(total was 756, now 763). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m 46s
{color} | {color:red} Patch causes 17 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 30s
{color} | {color:red} Patch causes 17 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 15s
{color} | {color:red} Patch causes 17 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 59s
{color} | {color:red} Patch causes 17 errors
