[jira] [Commented] (HBASE-17045) Unify the implementation of small scan and regular scan
[
https://issues.apache.org/jira/browse/HBASE-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827562#comment-15827562
]
Hadoop QA commented on HBASE-17045:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m 59s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
36s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 8s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 45s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 28s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 25s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
52s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 174m 41s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847977/HBASE-17045-v4.patch |
| JIRA Issue | HBASE-17045 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux b1535de1a9e3 3.13.0-98-generic #145-Ubuntu SMP Sat Oct 8
20:13:07 UTC 2016 x86_64 x86_6
[jira] [Commented] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
[
https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827557#comment-15827557
]
Yu Li commented on HBASE-17482:
---
Nice catch [~allan163], +1 on the fix.
Just one minor comment, that since the UT case is not that straight forward,
mind adding some javadoc for it, or at least add a link to the this JIRA?
Some clarification for the UT design I could think of (please correct me if I
state anything wrong):
After binding mvcc with sequence id, we will get cell's mvcc through
{{Cell#getSequenceId}} method, and will compare the mvcc with read point in
either {{SegmentScanner#updateCurrent}} (master branch) or
{{ScanQueryMatcher#match}} (branch-1) to make sure of read-write concurrency
control. And if we append cell into Memstore before its sequence id got
stamped, scan will see the data it shouldn't have seen. And the UT case
reproduces this scenario.
> mvcc mechanism failed when using mvccPreAssign
> --
>
> Key: HBASE-17482
> URL: https://issues.apache.org/jira/browse/HBASE-17482
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0..
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-17482.patch, HBASE-17482.v2.patch
>
>
> If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to
> memstore before append thread can stamp seqid to them. The unit test shows
> everything.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
[ https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827546#comment-15827546 ] Allan Yang commented on HBASE-17482: the failed test org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileRollWriter and org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushCacheWhileScanning are passed locally > mvcc mechanism failed when using mvccPreAssign > -- > > Key: HBASE-17482 > URL: https://issues.apache.org/jira/browse/HBASE-17482 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.. >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Critical > Attachments: HBASE-17482.patch, HBASE-17482.v2.patch > > > If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to > memstore before append thread can stamp seqid to them. The unit test shows > everything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17084) Delete HMerge (dead code)
[ https://issues.apache.org/jira/browse/HBASE-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17084: --- Resolution: Duplicate Assignee: (was: Appy) Status: Resolved (was: Patch Available) This is addressed in HBASE-17470 > Delete HMerge (dead code) > - > > Key: HBASE-17084 > URL: https://issues.apache.org/jira/browse/HBASE-17084 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Trivial > Attachments: HBASE-17084.master.001.patch > > > HMerge isn't used anywhere. Can we delete it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
[
https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827488#comment-15827488
]
Hadoop QA commented on HBASE-17482:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 13s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 17s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 151m 35s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegion |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847974/HBASE-17482.patch |
| JIRA Issue | HBASE-17482 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 1645e5f3c6a6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 805d39f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5305/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5305/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5305/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5305/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> mvcc mechanism failed when using mvccPreAssign
> --
>
> Key: HBASE-17482
>
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827483#comment-15827483 ] ramkrishna.s.vasudevan commented on HBASE-17081: Will commit this unless objections. [~anastas] - you will fix this minor comment in this JIRA? > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0 > > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBASE-17081-V07.patch, HBASE-17081-V10.patch, HBASE-17081-V13.patch, > HBASE-17081-V14.patch, HBASE-17081-V15.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17396) Add first async admin impl and implement balance methods
[
https://issues.apache.org/jira/browse/HBASE-17396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827444#comment-15827444
]
Hadoop QA commented on HBASE-17396:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 56s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 82m 37s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 125m 4s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847972/HBASE-17396-v7.patch |
| JIRA Issue | HBASE-17396 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 085a287ffe39 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 805d39f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5304/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5304/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add first async admin impl and implemen
[jira] [Updated] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
[ https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-17482: --- Attachment: HBASE-17482.v2.patch move test to TestFromClientSide3 > mvcc mechanism failed when using mvccPreAssign > -- > > Key: HBASE-17482 > URL: https://issues.apache.org/jira/browse/HBASE-17482 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.. >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Critical > Attachments: HBASE-17482.patch, HBASE-17482.v2.patch > > > If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to > memstore before append thread can stamp seqid to them. The unit test shows > everything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17396) Add first async admin impl and implement balance methods
[ https://issues.apache.org/jira/browse/HBASE-17396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827405#comment-15827405 ] Duo Zhang commented on HBASE-17396: --- +1 on the latest patch. Left a small question on rb. Can be fixed on commit. > Add first async admin impl and implement balance methods > > > Key: HBASE-17396 > URL: https://issues.apache.org/jira/browse/HBASE-17396 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17396-v1.patch, HBASE-17396-v2.patch, > HBASE-17396-v3.patch, HBASE-17396-v4.patch, HBASE-17396-v5.patch, > HBASE-17396-v6.patch, HBASE-17396-v7.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17045) Unify the implementation of small scan and regular scan
[ https://issues.apache.org/jira/browse/HBASE-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827402#comment-15827402 ] Duo Zhang commented on HBASE-17045: --- We reuse the Scan object across different scan operations in the UT, and we may modify the Scan object during scan. The problem is gone after I modify the UT to always use a new Scan object every time. The root cause is that, the logic of the while loop in ClientScanner.loadCache is broken. We may still issue a next request even if the moreResultsInRegion is false. And now we will close the scanner automatically when moreResultsInRegion is false at RS side, so we will get an UnknownScannerException and reset the startRow of the Scan object and open a new scanner. If later we use this Scan object to get a scanner then we can not get the results we want as the startRow is changed. Will try to fix this when porting this feature to the sync client. [~stack] [~enis] Any concerns? Thanks. > Unify the implementation of small scan and regular scan > --- > > Key: HBASE-17045 > URL: https://issues.apache.org/jira/browse/HBASE-17045 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17045.patch, HBASE-17045-v1.patch, > HBASE-17045-v2.patch, HBASE-17045-v3.patch, HBASE-17045-v4.patch > > > See [~enis]'s comment in HBASE-16838 > https://issues.apache.org/jira/browse/HBASE-16838?focusedCommentId=15637803&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15637803 > But there is another scenario that we need small scan is that, we do not know > the stop row but we only want a small set of results. For example, in the > implementation of region locator, we will use small scan and set caching to 1 > as we only need one row. > So I think we need to add a new option(maybe called limit?) for the scan > object, and deprecate the small option. And the server side modification > should also be committed to branch-1 to simplify the logic of async client in > 2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17271) hbase-thrift QA tests only run one test
[ https://issues.apache.org/jira/browse/HBASE-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17271: --- Labels: build thrift (was: thrift) > hbase-thrift QA tests only run one test > --- > > Key: HBASE-17271 > URL: https://issues.apache.org/jira/browse/HBASE-17271 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > Labels: build, thrift > > From > https://builds.apache.org/job/PreCommit-HBASE-Build/4822/artifact/patchprocess/patch-unit-hbase-thrift.txt > , it is pretty clear that only one test was run - TestCallQueue > Even though the patch contains modified TestThriftHBaseServiceHandler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17045) Unify the implementation of small scan and regular scan
[ https://issues.apache.org/jira/browse/HBASE-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-17045: -- Attachment: HBASE-17045-v4.patch Fix the failed UT. > Unify the implementation of small scan and regular scan > --- > > Key: HBASE-17045 > URL: https://issues.apache.org/jira/browse/HBASE-17045 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17045.patch, HBASE-17045-v1.patch, > HBASE-17045-v2.patch, HBASE-17045-v3.patch, HBASE-17045-v4.patch > > > See [~enis]'s comment in HBASE-16838 > https://issues.apache.org/jira/browse/HBASE-16838?focusedCommentId=15637803&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15637803 > But there is another scenario that we need small scan is that, we do not know > the stop row but we only want a small set of results. For example, in the > implementation of region locator, we will use small scan and set caching to 1 > as we only need one row. > So I think we need to add a new option(maybe called limit?) for the scan > object, and deprecate the small option. And the server side modification > should also be committed to branch-1 to simplify the logic of async client in > 2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827390#comment-15827390 ] Hudson commented on HBASE-17470: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2338 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2338/]) HBASE-17470 Remove merge region code from region server (Stephen Yuan (syuanjiangdev: rev 805d39fca6b6be004d2c554cc5d4f76bf48bc01a) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * (edit) hbase-protocol-shaded/src/main/protobuf/Master.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionImpl.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin1.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MasterProtos.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/util/Merge.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DispatchMergingRegionsProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockNoopMasterServices.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeRequest.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterAndRegionObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransaction.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/util/HMerge.java * (edit) hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MasterProcedureProtos.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransactionFactory.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestDispatchMergingRegionsProcedure.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, > HBASE-17470.v2-master.patch, HBASE-17470.v3-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17474) Reduce frequency of NoSuchMethodException when calling setStoragePolicy()
[
https://issues.apache.org/jira/browse/HBASE-17474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827389#comment-15827389
]
Hudson commented on HBASE-17474:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2338 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2338/])
HBASE-17474 Reduce frequency of NoSuchMethodException when calling (liyu: rev
287f95a579ee95a40e0f3a0986a246d29718ee3b)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
> Reduce frequency of NoSuchMethodException when calling setStoragePolicy()
> -
>
> Key: HBASE-17474
> URL: https://issues.apache.org/jira/browse/HBASE-17474
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Yu Li
>Priority: Minor
> Attachments: HBASE-17474.patch
>
>
> Saw the following in test output:
> {code}
> 2017-01-15 09:37:53,383 WARN
> [StoreOpener-98f4c892b5bcd9188f06331426e710f9-1]
> regionserver.HRegionFileSystem(199): Failed to set storage policy of
> [/Users/tyu/trunk/hbase-
> server/target/test-data/e052b6ad-02e2-41bf-a806-d617900a8c22/TestStoreFileRefresherChore/data/default/testIsStale/98f4c892b5bcd9188f06331426e710f9/cf]
> to [HOT]
> java.lang.UnsupportedOperationException: Cannot find specified method
> setStoragePolicy
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:209)
> at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:197)
> at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:244)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5254)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:988)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:985)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> java.lang.String)
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:205)
> ... 11 more
> {code}
> setStoragePolicy() is not implemented by LocalFileSystem.
> We should reduce the frequency of the above log since it clutters test output.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17436) Add facility to provide more information for Other Regions seen on Master UI
[ https://issues.apache.org/jira/browse/HBASE-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17436: --- Labels: ui (was: ) > Add facility to provide more information for Other Regions seen on Master UI > > > Key: HBASE-17436 > URL: https://issues.apache.org/jira/browse/HBASE-17436 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > Labels: ui > > [~rpednekar] and I were looking at a case where the count displayed under > Other Regions was high (~1200). > Since the table page just maintains a count instead of List of region names, > it is very difficult for user to determine which regions belong to this > category. > We should provide facility to provide more details for this category (LOG, > JMX, etc). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
[ https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827386#comment-15827386 ] Ted Yu commented on HBASE-17482: lgtm TestFromClientSide takes 248s on Jenkins. Consider moving the test to TestFromClientSide3 which is shorter currently. > mvcc mechanism failed when using mvccPreAssign > -- > > Key: HBASE-17482 > URL: https://issues.apache.org/jira/browse/HBASE-17482 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.. >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Critical > Attachments: HBASE-17482.patch > > > If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to > memstore before append thread can stamp seqid to them. The unit test shows > everything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
[ https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-17482: --- Status: Patch Available (was: Open) > mvcc mechanism failed when using mvccPreAssign > -- > > Key: HBASE-17482 > URL: https://issues.apache.org/jira/browse/HBASE-17482 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.. >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Critical > Attachments: HBASE-17482.patch > > > If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to > memstore before append thread can stamp seqid to them. The unit test shows > everything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
[ https://issues.apache.org/jira/browse/HBASE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-17482: --- Attachment: HBASE-17482.patch > mvcc mechanism failed when using mvccPreAssign > -- > > Key: HBASE-17482 > URL: https://issues.apache.org/jira/browse/HBASE-17482 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.. >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Critical > Attachments: HBASE-17482.patch > > > If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to > memstore before append thread can stamp seqid to them. The unit test shows > everything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17482) mvcc mechanism failed when using mvccPreAssign
Allan Yang created HBASE-17482: -- Summary: mvcc mechanism failed when using mvccPreAssign Key: HBASE-17482 URL: https://issues.apache.org/jira/browse/HBASE-17482 Project: HBase Issue Type: Bug Affects Versions: 2.0.. Reporter: Allan Yang Assignee: Allan Yang Priority: Critical If mvccPreAssign and ASYNC_WAL is used, then cells may have been commited to memstore before append thread can stamp seqid to them. The unit test shows everything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17396) Add first async admin impl and implement balance methods
[ https://issues.apache.org/jira/browse/HBASE-17396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17396: --- Attachment: HBASE-17396-v7.patch > Add first async admin impl and implement balance methods > > > Key: HBASE-17396 > URL: https://issues.apache.org/jira/browse/HBASE-17396 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17396-v1.patch, HBASE-17396-v2.patch, > HBASE-17396-v3.patch, HBASE-17396-v4.patch, HBASE-17396-v5.patch, > HBASE-17396-v6.patch, HBASE-17396-v7.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827327#comment-15827327 ] chunhui shen commented on HBASE-16630: -- [~dvdreddy] The method of freeEntireBuckets should return the freed bytes, then add it to the variable 'bytesFreed' of BucketCache#freeSpace. I argee that the new introduced method BucketCache#freeEntireBuckets will help to free space for fragmentation scene and no effect on no-fragmentation scene. So, patch is fine to me, thanks > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630.patch, HBASE-16630-v2.patch, HBASE-16630-v3.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17059) backport HBASE-17039 to 1.3.1
[ https://issues.apache.org/jira/browse/HBASE-17059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827319#comment-15827319 ] Yu Li commented on HBASE-17059: --- [~mantonov] Just to confirm sir, that since 1.3.0 is released, is branch-1.3 ready to accept commits now? Thanks. > backport HBASE-17039 to 1.3.1 > - > > Key: HBASE-17059 > URL: https://issues.apache.org/jira/browse/HBASE-17059 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.0 >Reporter: Yu Li >Assignee: Yu Li > Fix For: 1.3.1 > > > Currently branch-1.3 codes are freezing for 1.3.0 release, need to backport > HBASE-17039 to 1.3.1 afterwards. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827291#comment-15827291 ] chunhui shen commented on HBASE-16630: -- bq. If its harmless, why commit it (smile)? I mean it's harmless for no-fragmentation scene after applying this patch. :) > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630.patch, HBASE-16630-v2.patch, HBASE-16630-v3.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17165) Add retry to LoadIncrementalHFiles tool
[ https://issues.apache.org/jira/browse/HBASE-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Grimes updated HBASE-17165: Attachment: (was: HBASE-17165.branch-1.001.patch) > Add retry to LoadIncrementalHFiles tool > --- > > Key: HBASE-17165 > URL: https://issues.apache.org/jira/browse/HBASE-17165 > Project: HBase > Issue Type: Improvement > Components: hbase, HFile, tooling >Affects Versions: 2.0.0, 1.2.3 >Reporter: Mike Grimes >Assignee: Mike Grimes >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17165.branch-1.2.001.patch, > HBASE-17165.branch-1.2.002.patch, HBASE-17165.branch-1.2.003.patch, > HBASE-17165.branch-1.2.004.patch, HBASE-17165.master.001.patch, > HBASE-17165.master.002.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > As using the LoadIncrementalHFiles tool with S3 as the filesystem is prone to > failing due to FileNotFoundExceptions due to inconsistency, simple, > configurable retry logic was added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12894) Upgrade Jetty to 9.2.6
[ https://issues.apache.org/jira/browse/HBASE-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827205#comment-15827205 ] Guang Yang commented on HBASE-12894: FWIW, we have been running the Jetty 9 REST for more than a month and it is more stable comparing to the Jetty 6 version. With Jetty 6, we had a memory leak issue (the heap dump pointed the problem to Jetty core), with which we had to work around by periodically restarting the REST daemon. > Upgrade Jetty to 9.2.6 > -- > > Key: HBASE-12894 > URL: https://issues.apache.org/jira/browse/HBASE-12894 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 0.98.0 >Reporter: Rick Hallihan >Assignee: Guang Yang >Priority: Critical > Labels: MicrosoftSupport > Fix For: 2.0.0 > > Attachments: dependency_list_after, dependency_list_before, > HBASE-12894_Jetty9_v0.patch, HBASE-12894_Jetty9_v10.patch, > HBASE-12894_Jetty9_v1.patch, HBASE-12894_Jetty9_v1.patch, > HBASE-12894_Jetty9_v2.patch, HBASE-12894_Jetty9_v3.patch, > HBASE-12894_Jetty9_v4.patch, HBASE-12894_Jetty9_v5.patch, > HBASE-12894_Jetty9_v6.patch, HBASE-12894_Jetty9_v7.patch, > HBASE-12894_Jetty9_v8.patch > > > The Jetty component that is used for the HBase Stargate REST endpoint is > version 6.1.26 and is fairly outdated. We recently had a customer inquire > about enabling cross-origin resource sharing (CORS) for the REST endpoint and > found that this older version does not include the necessary filter or > configuration options, highlighted at: > http://wiki.eclipse.org/Jetty/Feature/Cross_Origin_Filter > The Jetty project has had significant updates through versions 7, 8 and 9, > including a transition to be an Eclipse subproject, so updating to the latest > version may be non-trivial. The last update to the Jetty component in > https://issues.apache.org/jira/browse/HBASE-3377 was a minor version update > and did not require significant work. This update will include a package > namespace update so there will likely be a larger number of required changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-12894) Upgrade Jetty to 9.2.6
[ https://issues.apache.org/jira/browse/HBASE-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826651#comment-15826651 ] Guang Yang edited comment on HBASE-12894 at 1/18/17 1:25 AM: - Hi [~busbey], Sorry I was on paternity leave and just back to work today. I just went through the newly added packages and corrected the licenses, and rebased the patch against master. Could you help to take a look? Thanks. was (Author: yguang11): Hi [~busbey], Sorry I was on paternity leave and just be back today. I just went through the newly added packages and correct the licenses, and rebase the patch against master. Could you help to take a look? Thanks. > Upgrade Jetty to 9.2.6 > -- > > Key: HBASE-12894 > URL: https://issues.apache.org/jira/browse/HBASE-12894 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 0.98.0 >Reporter: Rick Hallihan >Assignee: Guang Yang >Priority: Critical > Labels: MicrosoftSupport > Fix For: 2.0.0 > > Attachments: dependency_list_after, dependency_list_before, > HBASE-12894_Jetty9_v0.patch, HBASE-12894_Jetty9_v10.patch, > HBASE-12894_Jetty9_v1.patch, HBASE-12894_Jetty9_v1.patch, > HBASE-12894_Jetty9_v2.patch, HBASE-12894_Jetty9_v3.patch, > HBASE-12894_Jetty9_v4.patch, HBASE-12894_Jetty9_v5.patch, > HBASE-12894_Jetty9_v6.patch, HBASE-12894_Jetty9_v7.patch, > HBASE-12894_Jetty9_v8.patch > > > The Jetty component that is used for the HBase Stargate REST endpoint is > version 6.1.26 and is fairly outdated. We recently had a customer inquire > about enabling cross-origin resource sharing (CORS) for the REST endpoint and > found that this older version does not include the necessary filter or > configuration options, highlighted at: > http://wiki.eclipse.org/Jetty/Feature/Cross_Origin_Filter > The Jetty project has had significant updates through versions 7, 8 and 9, > including a transition to be an Eclipse subproject, so updating to the latest > version may be non-trivial. The last update to the Jetty component in > https://issues.apache.org/jira/browse/HBASE-3377 was a minor version update > and did not require significant work. This update will include a package > namespace update so there will likely be a larger number of required changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17165) Add retry to LoadIncrementalHFiles tool
[ https://issues.apache.org/jira/browse/HBASE-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Grimes updated HBASE-17165: Attachment: HBASE-17165.branch-1.001.patch > Add retry to LoadIncrementalHFiles tool > --- > > Key: HBASE-17165 > URL: https://issues.apache.org/jira/browse/HBASE-17165 > Project: HBase > Issue Type: Improvement > Components: hbase, HFile, tooling >Affects Versions: 2.0.0, 1.2.3 >Reporter: Mike Grimes >Assignee: Mike Grimes >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17165.branch-1.001.patch, > HBASE-17165.branch-1.2.001.patch, HBASE-17165.branch-1.2.002.patch, > HBASE-17165.branch-1.2.003.patch, HBASE-17165.branch-1.2.004.patch, > HBASE-17165.master.001.patch, HBASE-17165.master.002.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > As using the LoadIncrementalHFiles tool with S3 as the filesystem is prone to > failing due to FileNotFoundExceptions due to inconsistency, simple, > configurable retry logic was added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16786) Procedure V2 - Move ZK-lock's uses to Procedure framework locks (LockProcedure)
[
https://issues.apache.org/jira/browse/HBASE-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827169#comment-15827169
]
Hadoop QA commented on HBASE-16786:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 29s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 15 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 38s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 47s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 47s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 47s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 54s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 40s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 26s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 15s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.6.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 1s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 48s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 12m 37s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.7.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 14m 24s
{color} | {color:red} The patch causes 17 errors with Hadoop v2.7.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 16m 8s
{color} | {color:red} The patch causes 17 errors with Hadoop v3.0.0-alpha1.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 30s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 49s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 50s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color
[jira] [Commented] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827159#comment-15827159 ] Hudson commented on HBASE-17469: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2337 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2337/]) HBASE-17469 Properly handle empty TableName in (tedyu: rev faa9f735ca67ee3a2e1a59d93b519421e97e940f) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: Ted Yu >Assignee: Manjunath Anand > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12894) Upgrade Jetty to 9.2.6
[
https://issues.apache.org/jira/browse/HBASE-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827150#comment-15827150
]
Hadoop QA commented on HBASE-12894:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
20s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source:
hbase-resource-bundle . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 14s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 6s
{color} | {color:red} hbase-thrift in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 6s
{color} | {color:red} hbase-rest in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 30s
{color} | {color:red} hbase-spark in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 10s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 23s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source:
hbase-resource-bundle . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 38s
{color} | {color:red} hbase-rest generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 7s
{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 44s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 26s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 28s
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s
{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
[jira] [Commented] (HBASE-17481) [C++] Changing cpplint to ignore line wrapping warnings
[
https://issues.apache.org/jira/browse/HBASE-17481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827142#comment-15827142
]
Hadoop QA commented on HBASE-17481:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 6s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} The patch generated 0 new + 434 unchanged - 1 fixed
= 434 total (was 435) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 11s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
3s {color} | {color:green} the patch passed {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 42s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:date2017-01-18 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847942/HBASE-17481.HBASE-14850.v1.patch
|
| JIRA Issue | HBASE-17481 |
| Optional Tests | shellcheck shelldocs |
| uname | Linux 3af78d9c55c0 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-14850 / fe69c81 |
| shellcheck | v0.4.5 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5301/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [C++] Changing cpplint to ignore line wrapping warnings
> ---
>
> Key: HBASE-17481
> URL: https://issues.apache.org/jira/browse/HBASE-17481
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sudeep Sunthankar
>Assignee: Sudeep Sunthankar
> Attachments: HBASE-17481.HBASE-14850.v1.patch
>
>
> We have changed the line wrapping to 100. cpplint should reflect the same to
> avoid unnecessary warnings.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17278) [C++] Cell Scanner Implementation to be used by ResultScanner
[
https://issues.apache.org/jira/browse/HBASE-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827141#comment-15827141
]
Hadoop QA commented on HBASE-17278:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 7s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} HBASE-14850 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} HBASE-14850 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
1s {color} | {color:green} the patch passed {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 57s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:date2017-01-18 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847936/HBASE-17278.HBASE-14850.v2.patch
|
| JIRA Issue | HBASE-17278 |
| Optional Tests | shellcheck shelldocs cc compile |
| uname | Linux 814e02b26e86 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-14850 / fe69c81 |
| shellcheck | v0.4.5 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5302/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [C++] Cell Scanner Implementation to be used by ResultScanner
> -
>
> Key: HBASE-17278
> URL: https://issues.apache.org/jira/browse/HBASE-17278
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sudeep Sunthankar
>Assignee: Sudeep Sunthankar
> Attachments: HBASE-17278.HBASE-14850.v1.patch,
> HBASE-17278.HBASE-14850.v2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16786) Procedure V2 - Move ZK-lock's uses to Procedure framework locks (LockProcedure)
[ https://issues.apache.org/jira/browse/HBASE-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827115#comment-15827115 ] stack commented on HBASE-16786: --- -010 fixes unit tests by making it possible for MOB to obtain lock remotely (MOB will fail to get lock if Master is down). Had to change TestMobStoreCompaction#testMajorCompactionAfterDelete assert counts because runs with compaction.getRequest().forceRetainDeleteMarkers() if there is no regionServerServices available to get lock from. > Procedure V2 - Move ZK-lock's uses to Procedure framework locks > (LockProcedure) > --- > > Key: HBASE-16786 > URL: https://issues.apache.org/jira/browse/HBASE-16786 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Matteo Bertozzi > Attachments: HBASE-16786.master.001.patch, > HBASE-16786.master.002.patch, HBASE-16786.master.003.patch, > HBASE-16786.master.004.patch, HBASE-16786.master.005.patch, > HBASE-16786.master.006.patch, HBASE-16786.master.007.patch, > HBASE-16786.master.008.patch, HBASE-16786.master.009.patch, > HBASE-16786.master.010.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16786) Procedure V2 - Move ZK-lock's uses to Procedure framework locks (LockProcedure)
[ https://issues.apache.org/jira/browse/HBASE-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16786: -- Attachment: HBASE-16786.master.010.patch > Procedure V2 - Move ZK-lock's uses to Procedure framework locks > (LockProcedure) > --- > > Key: HBASE-16786 > URL: https://issues.apache.org/jira/browse/HBASE-16786 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Matteo Bertozzi > Attachments: HBASE-16786.master.001.patch, > HBASE-16786.master.002.patch, HBASE-16786.master.003.patch, > HBASE-16786.master.004.patch, HBASE-16786.master.005.patch, > HBASE-16786.master.006.patch, HBASE-16786.master.007.patch, > HBASE-16786.master.008.patch, HBASE-16786.master.009.patch, > HBASE-16786.master.010.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17481) [C++] Changing cpplint to ignore line wrapping warnings
[ https://issues.apache.org/jira/browse/HBASE-17481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar updated HBASE-17481: -- Status: Patch Available (was: Open) > [C++] Changing cpplint to ignore line wrapping warnings > --- > > Key: HBASE-17481 > URL: https://issues.apache.org/jira/browse/HBASE-17481 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17481.HBASE-14850.v1.patch > > > We have changed the line wrapping to 100. cpplint should reflect the same to > avoid unnecessary warnings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17481) [C++] Changing cpplint to ignore line wrapping warnings
[ https://issues.apache.org/jira/browse/HBASE-17481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar updated HBASE-17481: -- Attachment: HBASE-17481.HBASE-14850.v1.patch Added option --linelength=100 > [C++] Changing cpplint to ignore line wrapping warnings > --- > > Key: HBASE-17481 > URL: https://issues.apache.org/jira/browse/HBASE-17481 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17481.HBASE-14850.v1.patch > > > We have changed the line wrapping to 100. cpplint should reflect the same to > avoid unnecessary warnings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-17481) [C++] Changing cpplint to ignore line wrapping warnings
[ https://issues.apache.org/jira/browse/HBASE-17481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar reassigned HBASE-17481: - Assignee: Sudeep Sunthankar > [C++] Changing cpplint to ignore line wrapping warnings > --- > > Key: HBASE-17481 > URL: https://issues.apache.org/jira/browse/HBASE-17481 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > > We have changed the line wrapping to 100. cpplint should reflect the same to > avoid unnecessary warnings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17481) [C++] Changing cpplint to ignore line wrapping warnings
Sudeep Sunthankar created HBASE-17481: - Summary: [C++] Changing cpplint to ignore line wrapping warnings Key: HBASE-17481 URL: https://issues.apache.org/jira/browse/HBASE-17481 Project: HBase Issue Type: Sub-task Reporter: Sudeep Sunthankar We have changed the line wrapping to 100. cpplint should reflect the same to avoid unnecessary warnings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17460) enable_table_replication can not perform cyclic replication of a table
[
https://issues.apache.org/jira/browse/HBASE-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827078#comment-15827078
]
Hadoop QA commented on HBASE-17460:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 56s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 18s
{color} | {color:red} hbase-client generated 1 new + 13 unchanged - 0 fixed =
14 total (was 13) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 12s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 32s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847914/HBASE-17460_v2.patch |
| JIRA Issue | HBASE-17460 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e3743c022bd8 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / faa9f73 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/5300/artifact/patchprocess/whitespace-eol.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/5300/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5300/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5300/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> enable_table_r
[jira] [Updated] (HBASE-17474) Reduce frequency of NoSuchMethodException when calling setStoragePolicy()
[
https://issues.apache.org/jira/browse/HBASE-17474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-17474:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed into master branch. Thanks [~tedyu] for finding the issue and help
review.
> Reduce frequency of NoSuchMethodException when calling setStoragePolicy()
> -
>
> Key: HBASE-17474
> URL: https://issues.apache.org/jira/browse/HBASE-17474
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Yu Li
>Priority: Minor
> Attachments: HBASE-17474.patch
>
>
> Saw the following in test output:
> {code}
> 2017-01-15 09:37:53,383 WARN
> [StoreOpener-98f4c892b5bcd9188f06331426e710f9-1]
> regionserver.HRegionFileSystem(199): Failed to set storage policy of
> [/Users/tyu/trunk/hbase-
> server/target/test-data/e052b6ad-02e2-41bf-a806-d617900a8c22/TestStoreFileRefresherChore/data/default/testIsStale/98f4c892b5bcd9188f06331426e710f9/cf]
> to [HOT]
> java.lang.UnsupportedOperationException: Cannot find specified method
> setStoragePolicy
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:209)
> at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:197)
> at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:244)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5254)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:988)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:985)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> java.lang.String)
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:205)
> ... 11 more
> {code}
> setStoragePolicy() is not implemented by LocalFileSystem.
> We should reduce the frequency of the above log since it clutters test output.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17278) [C++] Cell Scanner Implementation to be used by ResultScanner
[ https://issues.apache.org/jira/browse/HBASE-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar updated HBASE-17278: -- Status: Patch Available (was: In Progress) > [C++] Cell Scanner Implementation to be used by ResultScanner > - > > Key: HBASE-17278 > URL: https://issues.apache.org/jira/browse/HBASE-17278 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17278.HBASE-14850.v1.patch, > HBASE-17278.HBASE-14850.v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17278) [C++] Cell Scanner Implementation to be used by ResultScanner
[ https://issues.apache.org/jira/browse/HBASE-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar updated HBASE-17278: -- Attachment: HBASE-17278.HBASE-14850.v2.patch In this patch we have the foll:- 1) Cell Scanner class which will iterate through sequence of cells obtained from cell_meta block 2) KeyValueCodec class where the actual logic of parsing is present Thanks. > [C++] Cell Scanner Implementation to be used by ResultScanner > - > > Key: HBASE-17278 > URL: https://issues.apache.org/jira/browse/HBASE-17278 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17278.HBASE-14850.v1.patch, > HBASE-17278.HBASE-14850.v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17470) Remove merge region code from region server
[
https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827025#comment-15827025
]
Hadoop QA commented on HBASE-17470:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 7s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 7 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 24s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 33s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 4s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 24s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 171m 45s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847897/HBASE-17470.v3-master.patch
|
| JIRA Issue | HBASE-17470 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux 970ca09b7f1d 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreC
[jira] [Updated] (HBASE-17480) Remove split region code from Region Server
[ https://issues.apache.org/jira/browse/HBASE-17480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17480: --- Description: HBASE-14551 moves the split region logic to the master-side. There is no need to keep region_server-side split region code. We should remove them to avoid code duplication. was: HBASE-16118 moves the split region logic to the master-side. There is no need to keep region_server-side split region code. We should remove them to avoid code duplication. > Remove split region code from Region Server > --- > > Key: HBASE-17480 > URL: https://issues.apache.org/jira/browse/HBASE-17480 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > > HBASE-14551 moves the split region logic to the master-side. There is no need > to keep region_server-side split region code. We should remove them to avoid > code duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17480) Remove split region code from Region Server
Stephen Yuan Jiang created HBASE-17480: -- Summary: Remove split region code from Region Server Key: HBASE-17480 URL: https://issues.apache.org/jira/browse/HBASE-17480 Project: HBase Issue Type: Sub-task Components: Region Assignment Affects Versions: 2.0.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang Fix For: 2.0.0 HBASE-16118 moves the split region logic to the master-side. There is no need to keep region_server-side split region code. We should remove them to avoid code duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10699) Optimize some code
[ https://issues.apache.org/jira/browse/HBASE-10699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827007#comment-15827007 ] Appy commented on HBASE-10699: -- Hi [~Jan Hentschel], welcome to the hbase community. It's great to see new contributions coming in. Looks like you have 12 patches pending for review, and most are < 10 lines. It'd be much easier to review and commit to repo if it was all in one single patch. Can you please do so? Thanks. > Optimize some code > -- > > Key: HBASE-10699 > URL: https://issues.apache.org/jira/browse/HBASE-10699 > Project: HBase > Issue Type: Improvement >Reporter: @deprecated Yi Deng >Assignee: Jan Hentschel > > 1. Set capacity on ArrayList when possible > 2. Use isEmpty instead of size() == 0 when possible > 3. Reduce some unnecessary allocation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17479) Add one more unittest case about supporting customer filters over hbase REST
huaxiang sun created HBASE-17479:
Summary: Add one more unittest case about supporting customer
filters over hbase REST
Key: HBASE-17479
URL: https://issues.apache.org/jira/browse/HBASE-17479
Project: HBase
Issue Type: Improvement
Reporter: huaxiang sun
Assignee: huaxiang sun
Priority: Minor
Plan to add another unittest case and update hbase book about how to use
customer filters over hbase REST.
{code}
+conf.set(Constants.CUSTOM_FILTERS, "CustomSingleColumnValueFilter:" +
CustomSingleColumnValueFilter.class.getName());
+
+ @Test
+ public void testCustomSingleColumnValueFilter() throws IOException,
JAXBException {
+StringBuilder builder = new StringBuilder();
+builder = new StringBuilder();
+builder.append("/*");
+builder.append("?");
+builder.append(Constants.SCAN_FILTER + "=" +
+URLEncoder.encode("CustomSingleColumnValueFilter('a', '1', =,
'binary:abc')", "UTF-8"));
+Response response =
+client.get("/" + TABLE + builder.toString(), Constants.MIMETYPE_XML);
+assertEquals(200, response.getCode());
+JAXBContext ctx = JAXBContext.newInstance(CellSetModel.class);
+Unmarshaller ush = ctx.createUnmarshaller();
+CellSetModel model = (CellSetModel) ush.unmarshal(response.getStream());
+int count = TestScannerResource.countCellSet(model);
+assertEquals(1, count);
+assertEquals("abc", new
String(model.getRows().get(0).getCells().get(0).getValue()));
+ }
+
+ public static class CustomSingleColumnValueFilter extends
SingleColumnValueFilter {
+
+public CustomSingleColumnValueFilter(final byte [] family, final byte []
qualifier,
+final CompareOp compareOp,
+final org.apache.hadoop.hbase.filter.ByteArrayComparable comparator) {
+ super(family, qualifier, compareOp, comparator);
+}
+
+
+public static Filter createFilterFromArguments(ArrayList
filterArguments) {
+ Preconditions.checkArgument(filterArguments.size() == 4 ||
filterArguments.size() == 6,
+ "Expected 4 or 6 but got: %s", filterArguments.size());
+ byte [] family =
ParseFilter.removeQuotesFromByteArray(filterArguments.get(0));
+ byte [] qualifier =
ParseFilter.removeQuotesFromByteArray(filterArguments.get(1));
+ CompareOp compareOp =
ParseFilter.createCompareOp(filterArguments.get(2));
+ org.apache.hadoop.hbase.filter.ByteArrayComparable comparator =
ParseFilter.createComparator(
+ ParseFilter.removeQuotesFromByteArray(filterArguments.get(3)));
+
+ if (comparator instanceof RegexStringComparator ||
+ comparator instanceof SubstringComparator) {
+if (compareOp != CompareOp.EQUAL &&
+compareOp != CompareOp.NOT_EQUAL) {
+ throw new IllegalArgumentException ("A regexstring comparator and
substring comparator " +
+ "can only be used with EQUAL and NOT_EQUAL");
+}
+ }
+
+ CustomSingleColumnValueFilter filter = new
CustomSingleColumnValueFilter(family, qualifier,
+ compareOp, comparator);
+
+ if (filterArguments.size() == 6) {
+boolean filterIfMissing =
ParseFilter.convertByteArrayToBoolean(filterArguments.get(4));
+boolean latestVersionOnly =
ParseFilter.convertByteArrayToBoolean(filterArguments.get(5));
+filter.setFilterIfMissing(filterIfMissing);
+filter.setLatestVersionOnly(latestVersionOnly);
+ }
+ return filter;
+}
+ }
+
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17460) enable_table_replication can not perform cyclic replication of a table
[ https://issues.apache.org/jira/browse/HBASE-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NITIN VERMA updated HBASE-17460: Status: Patch Available (was: In Progress) > enable_table_replication can not perform cyclic replication of a table > -- > > Key: HBASE-17460 > URL: https://issues.apache.org/jira/browse/HBASE-17460 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: NITIN VERMA >Assignee: NITIN VERMA > Labels: replication > Attachments: HBASE-17460.patch, HBASE-17460_v2.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > The enable_table_replication operation is broken for cyclic replication of > HBase table as we compare all the properties of column families (including > REPLICATION_SCOPE). > Below is exactly what happens: > 1. Running "enable_table_replication 'table1' " opeartion on first cluster > will set the REPLICATION_SCOPE of all column families to peer id '1'. This > will also create a table on second cluster where REPLICATION_SCOPE is still > set to peer id '0'. > 2. Now when we run "enable_table_replication 'table1'" on second cluster, we > compare all the properties of table (including REPLICATION_SCOPE_, which > obviously is different now. > I am proposing a fix for this issue where we should avoid comparing > REPLICATION_SCOPE inside HColumnDescriotor::compareTo() method, especially > when replication is not already enabled on the desired table. > I have made that change and it is working. I will submit the patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17460) enable_table_replication can not perform cyclic replication of a table
[ https://issues.apache.org/jira/browse/HBASE-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NITIN VERMA updated HBASE-17460: Status: Patch Available (was: In Progress) HBASE-17460_v2.patch is available for review. I have tested the functionality on my local HBase replication setup. Below are the scenarios I have tested: 1. Ensure enable_table_replication can perform cyclic replication. 2. Ensure enable_table_replication fails on a table where replication has already been enabled. > enable_table_replication can not perform cyclic replication of a table > -- > > Key: HBASE-17460 > URL: https://issues.apache.org/jira/browse/HBASE-17460 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: NITIN VERMA >Assignee: NITIN VERMA > Labels: replication > Attachments: HBASE-17460.patch, HBASE-17460_v2.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > The enable_table_replication operation is broken for cyclic replication of > HBase table as we compare all the properties of column families (including > REPLICATION_SCOPE). > Below is exactly what happens: > 1. Running "enable_table_replication 'table1' " opeartion on first cluster > will set the REPLICATION_SCOPE of all column families to peer id '1'. This > will also create a table on second cluster where REPLICATION_SCOPE is still > set to peer id '0'. > 2. Now when we run "enable_table_replication 'table1'" on second cluster, we > compare all the properties of table (including REPLICATION_SCOPE_, which > obviously is different now. > I am proposing a fix for this issue where we should avoid comparing > REPLICATION_SCOPE inside HColumnDescriotor::compareTo() method, especially > when replication is not already enabled on the desired table. > I have made that change and it is working. I will submit the patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17460) enable_table_replication can not perform cyclic replication of a table
[ https://issues.apache.org/jira/browse/HBASE-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NITIN VERMA updated HBASE-17460: Status: In Progress (was: Patch Available) > enable_table_replication can not perform cyclic replication of a table > -- > > Key: HBASE-17460 > URL: https://issues.apache.org/jira/browse/HBASE-17460 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: NITIN VERMA >Assignee: NITIN VERMA > Labels: replication > Attachments: HBASE-17460.patch, HBASE-17460_v2.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > The enable_table_replication operation is broken for cyclic replication of > HBase table as we compare all the properties of column families (including > REPLICATION_SCOPE). > Below is exactly what happens: > 1. Running "enable_table_replication 'table1' " opeartion on first cluster > will set the REPLICATION_SCOPE of all column families to peer id '1'. This > will also create a table on second cluster where REPLICATION_SCOPE is still > set to peer id '0'. > 2. Now when we run "enable_table_replication 'table1'" on second cluster, we > compare all the properties of table (including REPLICATION_SCOPE_, which > obviously is different now. > I am proposing a fix for this issue where we should avoid comparing > REPLICATION_SCOPE inside HColumnDescriotor::compareTo() method, especially > when replication is not already enabled on the desired table. > I have made that change and it is working. I will submit the patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826970#comment-15826970 ] stack commented on HBASE-17470: --- Sounds good [~syuanjiang] > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, > HBASE-17470.v2-master.patch, HBASE-17470.v3-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826957#comment-15826957 ] Stephen Yuan Jiang commented on HBASE-17470: [~stack], already did that in release notes. I am waiting for pre-commit to run the V3 patch before commit. > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, > HBASE-17470.v2-master.patch, HBASE-17470.v3-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826944#comment-15826944 ] Hudson commented on HBASE-17469: FAILURE: Integrated in Jenkins build HBase-1.4 #595 (See [https://builds.apache.org/job/HBase-1.4/595/]) HBASE-17469 Properly handle empty TableName in (tedyu: rev 0e06ade694e8bfd23898c958d219470ae89d1d32) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: Ted Yu >Assignee: Manjunath Anand > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826921#comment-15826921 ] stack commented on HBASE-17470: --- Thanks [~syuanjiang] Sounds good. Maybe hoist your comment up into release notes section on this issue and then commit. +1. > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, > HBASE-17470.v2-master.patch, HBASE-17470.v3-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17460) enable_table_replication can not perform cyclic replication of a table
[ https://issues.apache.org/jira/browse/HBASE-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NITIN VERMA updated HBASE-17460: Attachment: HBASE-17460_v2.patch Besides addressing comments from [~ashish singhi] and [~enis], I have also taken care of following scenario: Today, when replication is enabled already on a table, the enable_table_replication command succeeds again. Ideally it should throw an error saying replication has already been enabled on the table. The REPLICATION_SCOPE for a CF could be set to any peer (with id > 1) and running enable_table_replication on that table should fail. > enable_table_replication can not perform cyclic replication of a table > -- > > Key: HBASE-17460 > URL: https://issues.apache.org/jira/browse/HBASE-17460 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: NITIN VERMA >Assignee: NITIN VERMA > Labels: replication > Attachments: HBASE-17460.patch, HBASE-17460_v2.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > The enable_table_replication operation is broken for cyclic replication of > HBase table as we compare all the properties of column families (including > REPLICATION_SCOPE). > Below is exactly what happens: > 1. Running "enable_table_replication 'table1' " opeartion on first cluster > will set the REPLICATION_SCOPE of all column families to peer id '1'. This > will also create a table on second cluster where REPLICATION_SCOPE is still > set to peer id '0'. > 2. Now when we run "enable_table_replication 'table1'" on second cluster, we > compare all the properties of table (including REPLICATION_SCOPE_, which > obviously is different now. > I am proposing a fix for this issue where we should avoid comparing > REPLICATION_SCOPE inside HColumnDescriotor::compareTo() method, especially > when replication is not already enabled on the desired table. > I have made that change and it is working. I will submit the patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17460) enable_table_replication can not perform cyclic replication of a table
[ https://issues.apache.org/jira/browse/HBASE-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NITIN VERMA updated HBASE-17460: Status: In Progress (was: Patch Available) Example of new functionality added: --- hbase(main):014:0> enable_table_replication 'table1' The replication of table 'table1' successfully enabled Took 0.5180 seconds hbase(main):016:0> enable_table_replication 'table1' ERROR: Table table1 has replication already enabled for atleast one Column Family. Here is some help for this command: Enable a table's replication switch. Examples: hbase> enable_table_replication 'table_name' Took 0.1180 seconds > enable_table_replication can not perform cyclic replication of a table > -- > > Key: HBASE-17460 > URL: https://issues.apache.org/jira/browse/HBASE-17460 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: NITIN VERMA >Assignee: NITIN VERMA > Labels: replication > Attachments: HBASE-17460.patch, HBASE-17460_v2.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > The enable_table_replication operation is broken for cyclic replication of > HBase table as we compare all the properties of column families (including > REPLICATION_SCOPE). > Below is exactly what happens: > 1. Running "enable_table_replication 'table1' " opeartion on first cluster > will set the REPLICATION_SCOPE of all column families to peer id '1'. This > will also create a table on second cluster where REPLICATION_SCOPE is still > set to peer id '0'. > 2. Now when we run "enable_table_replication 'table1'" on second cluster, we > compare all the properties of table (including REPLICATION_SCOPE_, which > obviously is different now. > I am proposing a fix for this issue where we should avoid comparing > REPLICATION_SCOPE inside HColumnDescriotor::compareTo() method, especially > when replication is not already enabled on the desired table. > I have made that change and it is working. I will submit the patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17445) Count size of serialized exceptions in checking max result size quota
[ https://issues.apache.org/jira/browse/HBASE-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826836#comment-15826836 ] Hudson commented on HBASE-17445: FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #95 (See [https://builds.apache.org/job/HBase-1.3-JDK8/95/]) HBASE-17445 Count size of serialized exceptions in checking max result (tedyu: rev dc72e0d4f802fd0f4902e4552e3c0eada1c49d51) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcCallContext.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java > Count size of serialized exceptions in checking max result size quota > - > > Key: HBASE-17445 > URL: https://issues.apache.org/jira/browse/HBASE-17445 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: 17445.branch-1.v2.txt, 17445.v1.txt, 17445.v2.txt > > > Currently RSRpcServices#doNonAtomicRegionMutation() does accounting for cell > size and block size against the max result size quota. > However, as HBASE-17387 and related JIRAs have shown, we should consider the > size of exceptions sent back to the client as well. > This issue adds accounting for size of serialized exceptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17387) Reduce the overhead of exception report in RegionActionResult for multi()
[
https://issues.apache.org/jira/browse/HBASE-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826837#comment-15826837
]
Hudson commented on HBASE-17387:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #95 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/95/])
HBASE-17387 Reduce the overhead of exception report in (tedyu: rev
eb19a67486e1d722ce2f9d3484c0a7b111438d43)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
> Reduce the overhead of exception report in RegionActionResult for multi()
> -
>
> Key: HBASE-17387
> URL: https://issues.apache.org/jira/browse/HBASE-17387
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: 17387.v1.txt, 17387.v2.txt, 17387.v3.txt
>
>
> For RSRpcServices#doNonAtomicRegionMutation() :
> {code}
> for (ClientProtos.Action action: actions.getActionList()) {
> ...
> } catch (IOException ie) {
> rpcServer.getMetrics().exception(ie);
> resultOrExceptionBuilder = ResultOrException.newBuilder().
> setException(ResponseConverter.buildException(ie));
> }
> if (resultOrExceptionBuilder != null) {
> // Propagate index.
> resultOrExceptionBuilder.setIndex(action.getIndex());
> builder.addResultOrException(resultOrExceptionBuilder.build());
> }
> {code}
> The exceptions are added to builder in the for loop.
> The ClientProtos.ResultOrException.Builder instance is created within the for
> loop.
> For large multi call, this may incur non-trivial overhead for garbage
> collector if there're many exceptions.
> e.g. Here was sample debug log showing the actions in a batch:
> {code}
> 2016-12-23 04:21:56,263 DEBUG
> org.apache.hadoop.hbase.regionserver.RSRpcServices: NonAtomicRegionMutation
> batch summary: numAppends=0, numDeletes=11, numGets=0, numIncrements=15638,
> numPuts=15627, numServiceCalls=0, serializedSize=3871713,
> user=hbase/pob3.G.COM (auth:KERBEROS), client=null
> {code}
> Note the large number of increments in the batch.
> When the increments encounter conflict at server side, the overhead of
> stringified exceptions in response is considerable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17470: --- Release Note: In 1.x branches, Admin.mergeRegions calls MASTER via dispatchMergingRegions RPC; when executing dispatchMergingRegions RPC, MASTER calls RS via MergeRegions to complete the merge in RS-side. With HBASE-16119, the merge logic moves to master-side. This JIRA cleans up unused RPCs (dispatchMergingRegions and MergeRegions) , removes dangerous tools such as Merge and HMerge, and deletes unused RegionServer-side merge region logic in 2.0 release. > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, > HBASE-17470.v2-master.patch, HBASE-17470.v3-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826744#comment-15826744 ] Stephen Yuan Jiang commented on HBASE-17470: [~stack], the dispatchMergingRegions RPC exists in 1.x branches too. It was used to call DispatchMergingRegionHandler in master and then master send MergeRegionsRequest call to RS to complete merge in RS-side. In master branch, the DispatchMergingRegionHandler was replaced by DispatchMergingRegionsProcedure. Now MergeTableRegionsProcedure moves all the logic to master and we no longer need those RPCs. The incompatible changes are about removing these RPCs and remove Merge and HMerge tools in master branch. For the testing, since only internal implementation is changed, any existing tests that calls Admin.mergeRegions and Admin.mergeRegionsAsync would still be there to cover this feature (and also MergeTableRegionsProcedure has UT to cover merge in component level). By the way, before removing these RS-side of code, HBASE-16119 already made all merge code to call the new MergeTableRegionsProcedure (aka, the RS-side of logic already obsolete with the HBASE-16119 commit; I did miss one in TestAdmin1, which I fixed in this patch, probably the only real change in the patch). > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, > HBASE-17470.v2-master.patch, HBASE-17470.v3-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-17470: --- Attachment: HBASE-17470.v3-master.patch > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, > HBASE-17470.v2-master.patch, HBASE-17470.v3-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17001) [RegionServer] Implement enforcement of quota violation policies
[
https://issues.apache.org/jira/browse/HBASE-17001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826729#comment-15826729
]
Hadoop QA commented on HBASE-17001:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s {color}
| {color:red} HBASE-17001 does not apply to HBASE-16961. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847895/HBASE-17001.004.HBASE-16961.patch
|
| JIRA Issue | HBASE-17001 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5298/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [RegionServer] Implement enforcement of quota violation policies
>
>
> Key: HBASE-17001
> URL: https://issues.apache.org/jira/browse/HBASE-17001
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17001.001.patch, HBASE-17001.003.patch,
> HBASE-17001.004.HBASE-16961.patch
>
>
> When the master enacts a quota violation policy, the RegionServers need to
> actually enforce that policy per its definition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17445) Count size of serialized exceptions in checking max result size quota
[ https://issues.apache.org/jira/browse/HBASE-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826720#comment-15826720 ] Hudson commented on HBASE-17445: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #82 (See [https://builds.apache.org/job/HBase-1.3-JDK7/82/]) HBASE-17445 Count size of serialized exceptions in checking max result (tedyu: rev dc72e0d4f802fd0f4902e4552e3c0eada1c49d51) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcCallContext.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java > Count size of serialized exceptions in checking max result size quota > - > > Key: HBASE-17445 > URL: https://issues.apache.org/jira/browse/HBASE-17445 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: 17445.branch-1.v2.txt, 17445.v1.txt, 17445.v2.txt > > > Currently RSRpcServices#doNonAtomicRegionMutation() does accounting for cell > size and block size against the max result size quota. > However, as HBASE-17387 and related JIRAs have shown, we should consider the > size of exceptions sent back to the client as well. > This issue adds accounting for size of serialized exceptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17387) Reduce the overhead of exception report in RegionActionResult for multi()
[
https://issues.apache.org/jira/browse/HBASE-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826721#comment-15826721
]
Hudson commented on HBASE-17387:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #82 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/82/])
HBASE-17387 Reduce the overhead of exception report in (tedyu: rev
eb19a67486e1d722ce2f9d3484c0a7b111438d43)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
> Reduce the overhead of exception report in RegionActionResult for multi()
> -
>
> Key: HBASE-17387
> URL: https://issues.apache.org/jira/browse/HBASE-17387
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: 17387.v1.txt, 17387.v2.txt, 17387.v3.txt
>
>
> For RSRpcServices#doNonAtomicRegionMutation() :
> {code}
> for (ClientProtos.Action action: actions.getActionList()) {
> ...
> } catch (IOException ie) {
> rpcServer.getMetrics().exception(ie);
> resultOrExceptionBuilder = ResultOrException.newBuilder().
> setException(ResponseConverter.buildException(ie));
> }
> if (resultOrExceptionBuilder != null) {
> // Propagate index.
> resultOrExceptionBuilder.setIndex(action.getIndex());
> builder.addResultOrException(resultOrExceptionBuilder.build());
> }
> {code}
> The exceptions are added to builder in the for loop.
> The ClientProtos.ResultOrException.Builder instance is created within the for
> loop.
> For large multi call, this may incur non-trivial overhead for garbage
> collector if there're many exceptions.
> e.g. Here was sample debug log showing the actions in a batch:
> {code}
> 2016-12-23 04:21:56,263 DEBUG
> org.apache.hadoop.hbase.regionserver.RSRpcServices: NonAtomicRegionMutation
> batch summary: numAppends=0, numDeletes=11, numGets=0, numIncrements=15638,
> numPuts=15627, numServiceCalls=0, serializedSize=3871713,
> user=hbase/pob3.G.COM (auth:KERBEROS), client=null
> {code}
> Note the large number of increments in the batch.
> When the increments encounter conflict at server side, the overhead of
> stringified exceptions in response is considerable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17001) [RegionServer] Implement enforcement of quota violation policies
[ https://issues.apache.org/jira/browse/HBASE-17001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17001: --- Attachment: HBASE-17001.004.HBASE-16961.patch .004 Test classification and removes bulk-import checks for the purposes of quotas happening when they shouldn't. > [RegionServer] Implement enforcement of quota violation policies > > > Key: HBASE-17001 > URL: https://issues.apache.org/jira/browse/HBASE-17001 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-17001.001.patch, HBASE-17001.003.patch, > HBASE-17001.004.HBASE-16961.patch > > > When the master enacts a quota violation policy, the RegionServers need to > actually enforce that policy per its definition. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17478) Avoid sending FSUtilization reports to master when quota support is not enabled
[ https://issues.apache.org/jira/browse/HBASE-17478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17478: --- Attachment: HBASE-17478.001.HBASE-16961.patch .001 Trivial little change. > Avoid sending FSUtilization reports to master when quota support is not > enabled > --- > > Key: HBASE-17478 > URL: https://issues.apache.org/jira/browse/HBASE-17478 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Trivial > Attachments: HBASE-17478.001.HBASE-16961.patch > > > Trivial little change to make sure that the RS's do not send the filesystem > utilization reports to the master when hbase.quota.enabled=false and, > similarly, that the master gracefully handles these reports when the feature > is not enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17478) Avoid sending FSUtilization reports to master when quota support is not enabled
Josh Elser created HBASE-17478: -- Summary: Avoid sending FSUtilization reports to master when quota support is not enabled Key: HBASE-17478 URL: https://issues.apache.org/jira/browse/HBASE-17478 Project: HBase Issue Type: Sub-task Reporter: Josh Elser Assignee: Josh Elser Priority: Trivial Trivial little change to make sure that the RS's do not send the filesystem utilization reports to the master when hbase.quota.enabled=false and, similarly, that the master gracefully handles these reports when the feature is not enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826660#comment-15826660 ] deepankar commented on HBASE-16630: --- Sorry I forgot about the unit test, will see If I hack get something up. About [~stack]'s suggestion it was somewhat orthogonal to this issue, I agree that the eviction could be improved via the suggestions based in HBASE-15560, but it would be hard to prevent the fragmentation from happening when we have buckets allocated to different sizes and buckets are occupied very sparsely, and these occupants are blocks which are accessed recently and also frequently. > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630.patch, HBASE-16630-v2.patch, HBASE-16630-v3.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12894) Upgrade Jetty to 9.2.6
[ https://issues.apache.org/jira/browse/HBASE-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826651#comment-15826651 ] Guang Yang commented on HBASE-12894: Hi [~busbey], Sorry I was on paternity leave and just be back today. I just went through the newly added packages and correct the licenses, and rebase the patch against master. Could you help to take a look? Thanks. > Upgrade Jetty to 9.2.6 > -- > > Key: HBASE-12894 > URL: https://issues.apache.org/jira/browse/HBASE-12894 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 0.98.0 >Reporter: Rick Hallihan >Assignee: Guang Yang >Priority: Critical > Labels: MicrosoftSupport > Fix For: 2.0.0 > > Attachments: dependency_list_after, dependency_list_before, > HBASE-12894_Jetty9_v0.patch, HBASE-12894_Jetty9_v10.patch, > HBASE-12894_Jetty9_v1.patch, HBASE-12894_Jetty9_v1.patch, > HBASE-12894_Jetty9_v2.patch, HBASE-12894_Jetty9_v3.patch, > HBASE-12894_Jetty9_v4.patch, HBASE-12894_Jetty9_v5.patch, > HBASE-12894_Jetty9_v6.patch, HBASE-12894_Jetty9_v7.patch, > HBASE-12894_Jetty9_v8.patch > > > The Jetty component that is used for the HBase Stargate REST endpoint is > version 6.1.26 and is fairly outdated. We recently had a customer inquire > about enabling cross-origin resource sharing (CORS) for the REST endpoint and > found that this older version does not include the necessary filter or > configuration options, highlighted at: > http://wiki.eclipse.org/Jetty/Feature/Cross_Origin_Filter > The Jetty project has had significant updates through versions 7, 8 and 9, > including a transition to be an Eclipse subproject, so updating to the latest > version may be non-trivial. The last update to the Jetty component in > https://issues.apache.org/jira/browse/HBASE-3377 was a minor version update > and did not require significant work. This update will include a package > namespace update so there will likely be a larger number of required changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work stopped] (HBASE-12894) Upgrade Jetty to 9.2.6
[ https://issues.apache.org/jira/browse/HBASE-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-12894 stopped by Guang Yang. -- > Upgrade Jetty to 9.2.6 > -- > > Key: HBASE-12894 > URL: https://issues.apache.org/jira/browse/HBASE-12894 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 0.98.0 >Reporter: Rick Hallihan >Assignee: Guang Yang >Priority: Critical > Labels: MicrosoftSupport > Fix For: 2.0.0 > > Attachments: dependency_list_after, dependency_list_before, > HBASE-12894_Jetty9_v0.patch, HBASE-12894_Jetty9_v10.patch, > HBASE-12894_Jetty9_v1.patch, HBASE-12894_Jetty9_v1.patch, > HBASE-12894_Jetty9_v2.patch, HBASE-12894_Jetty9_v3.patch, > HBASE-12894_Jetty9_v4.patch, HBASE-12894_Jetty9_v5.patch, > HBASE-12894_Jetty9_v6.patch, HBASE-12894_Jetty9_v7.patch, > HBASE-12894_Jetty9_v8.patch > > > The Jetty component that is used for the HBase Stargate REST endpoint is > version 6.1.26 and is fairly outdated. We recently had a customer inquire > about enabling cross-origin resource sharing (CORS) for the REST endpoint and > found that this older version does not include the necessary filter or > configuration options, highlighted at: > http://wiki.eclipse.org/Jetty/Feature/Cross_Origin_Filter > The Jetty project has had significant updates through versions 7, 8 and 9, > including a transition to be an Eclipse subproject, so updating to the latest > version may be non-trivial. The last update to the Jetty component in > https://issues.apache.org/jira/browse/HBASE-3377 was a minor version update > and did not require significant work. This update will include a package > namespace update so there will likely be a larger number of required changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12894) Upgrade Jetty to 9.2.6
[ https://issues.apache.org/jira/browse/HBASE-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guang Yang updated HBASE-12894: --- Status: Patch Available (was: Open) > Upgrade Jetty to 9.2.6 > -- > > Key: HBASE-12894 > URL: https://issues.apache.org/jira/browse/HBASE-12894 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 0.98.0 >Reporter: Rick Hallihan >Assignee: Guang Yang >Priority: Critical > Labels: MicrosoftSupport > Fix For: 2.0.0 > > Attachments: dependency_list_after, dependency_list_before, > HBASE-12894_Jetty9_v0.patch, HBASE-12894_Jetty9_v10.patch, > HBASE-12894_Jetty9_v1.patch, HBASE-12894_Jetty9_v1.patch, > HBASE-12894_Jetty9_v2.patch, HBASE-12894_Jetty9_v3.patch, > HBASE-12894_Jetty9_v4.patch, HBASE-12894_Jetty9_v5.patch, > HBASE-12894_Jetty9_v6.patch, HBASE-12894_Jetty9_v7.patch, > HBASE-12894_Jetty9_v8.patch > > > The Jetty component that is used for the HBase Stargate REST endpoint is > version 6.1.26 and is fairly outdated. We recently had a customer inquire > about enabling cross-origin resource sharing (CORS) for the REST endpoint and > found that this older version does not include the necessary filter or > configuration options, highlighted at: > http://wiki.eclipse.org/Jetty/Feature/Cross_Origin_Filter > The Jetty project has had significant updates through versions 7, 8 and 9, > including a transition to be an Eclipse subproject, so updating to the latest > version may be non-trivial. The last update to the Jetty component in > https://issues.apache.org/jira/browse/HBASE-3377 was a minor version update > and did not require significant work. This update will include a package > namespace update so there will likely be a larger number of required changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12894) Upgrade Jetty to 9.2.6
[ https://issues.apache.org/jira/browse/HBASE-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guang Yang updated HBASE-12894: --- Attachment: HBASE-12894_Jetty9_v10.patch > Upgrade Jetty to 9.2.6 > -- > > Key: HBASE-12894 > URL: https://issues.apache.org/jira/browse/HBASE-12894 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 0.98.0 >Reporter: Rick Hallihan >Assignee: Guang Yang >Priority: Critical > Labels: MicrosoftSupport > Fix For: 2.0.0 > > Attachments: dependency_list_after, dependency_list_before, > HBASE-12894_Jetty9_v0.patch, HBASE-12894_Jetty9_v10.patch, > HBASE-12894_Jetty9_v1.patch, HBASE-12894_Jetty9_v1.patch, > HBASE-12894_Jetty9_v2.patch, HBASE-12894_Jetty9_v3.patch, > HBASE-12894_Jetty9_v4.patch, HBASE-12894_Jetty9_v5.patch, > HBASE-12894_Jetty9_v6.patch, HBASE-12894_Jetty9_v7.patch, > HBASE-12894_Jetty9_v8.patch > > > The Jetty component that is used for the HBase Stargate REST endpoint is > version 6.1.26 and is fairly outdated. We recently had a customer inquire > about enabling cross-origin resource sharing (CORS) for the REST endpoint and > found that this older version does not include the necessary filter or > configuration options, highlighted at: > http://wiki.eclipse.org/Jetty/Feature/Cross_Origin_Filter > The Jetty project has had significant updates through versions 7, 8 and 9, > including a transition to be an Eclipse subproject, so updating to the latest > version may be non-trivial. The last update to the Jetty component in > https://issues.apache.org/jira/browse/HBASE-3377 was a minor version update > and did not require significant work. This update will include a package > namespace update so there will likely be a larger number of required changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17437) Support specifying a WAL directory outside of the root directory
[ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826585#comment-15826585 ] Zach York commented on HBASE-17437: --- [~busbey] Yes diff 2 of RB and master.004 are the same. > Support specifying a WAL directory outside of the root directory > > > Key: HBASE-17437 > URL: https://issues.apache.org/jira/browse/HBASE-17437 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration, wal >Affects Versions: 1.2.4 >Reporter: Yishan Yang >Assignee: Zach York > Labels: patch > Fix For: 2.0.0, 1.4.0 > > Attachments: hbase-17437-branch-1.2.patch, > HBASE-17437.master.001.patch, HBASE-17437.master.002.patch, > HBASE-17437.master.003.patch, HBASE-17437.master.004.patch, > hbase-17437-master.patch > > > Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some > FileSystems (such as Amazon S3) don’t support append or consistent writes. > These two properties are imperative for the WAL in order to avoid loss of > writes. However, StoreFiles don’t necessarily need the same consistency > guarantees (since writes are cached locally and if writes fail, they can > always be replayed from the WAL). > > This JIRA aims to allow users to configure a log directory (for WALs) that is > outside of the root directory or even in a different FileSystem. The default > value will still put the log directory under the root directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17476) FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server host04.byzoro.com,60020,1484273922270: org.apache.hadoop.hbase.YouAreDeadException: S
[
https://issues.apache.org/jira/browse/HBASE-17476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826583#comment-15826583
]
Appy commented on HBASE-17476:
--
There was a long GC (80+ sec) because of which RS was considered dead. Doesn't
look like there's any issue here.
> FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server host04.byzoro.com,60020,1484273922270:
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently pros dead server
>
>
> Key: HBASE-17476
> URL: https://issues.apache.org/jira/browse/HBASE-17476
> Project: HBase
> Issue Type: Bug
>Reporter: tingtingyuchao
>
> {noformat}
> GC pool 'ParNew' had collection(s): count=1 time=3237ms
> 2017-01-16 17:15:25,185 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor:
> Detected pause in JVM or host machine (eg GC): pause of approximately 1319ms
> GC pool 'ParNew' had collection(s): count=1 time=1697ms
> 2017-01-16 17:15:25,215 WARN org.apache.hadoop.hbase.ipc.RpcServer:
> (responseTooSlow):
> {"processingtimems":13043,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:59138","starttimems":1484558112172,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
> 2017-01-16 17:15:25,606 WARN org.apache.hadoop.hbase.ipc.RpcServer:
> (responseTooSlow):
> {"processingtimems":16061,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:59114","starttimems":1484558109545,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
> 2017-01-16 17:15:25,901 WARN org.apache.hadoop.hbase.ipc.RpcServer:
> (responseTooSlow):
> {"processingtimems":11495,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:60177","starttimems":1484558114405,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
> 2017-01-16 17:15:25,991 WARN org.apache.hadoop.hbase.ipc.RpcServer:
> (responseTooSlow):
> {"processingtimems":12069,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:59673","starttimems":1484558113922,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
> 2017-01-16 17:15:27,000 WARN org.apache.hadoop.hbase.ipc.RpcServer:
> (responseTooSlow):
> {"processingtimems":15320,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.226:39885","starttimems":1484558111679,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
> 2017-01-16 17:15:30,032 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor:
> Detected pause in JVM or host machine (eg GC): pause of approximately 2846ms
> GC pool 'ParNew' had collection(s): count=1 time=3025ms
> 2017-01-16 17:17:00,517 INFO org.apache.zookeeper.ClientCnxn: Client session
> timed out, have not heard from server in 98152ms for sessionid
> 0x458f196c5cbdbaf, closing socket connection and attempting reconnect
> 2017-01-16 17:17:00,517 INFO org.apache.zookeeper.ClientCnxn: Client session
> timed out, have not heard from server in 98152ms for sessionid
> 0x458f196c5cbdbad, closing socket connection and attempting reconnect
> 2017-01-16 17:17:00,517 WARN org.apache.hadoop.hbase.ipc.RpcServer:
> PriorityRpcServer.handler=3,queue=1,port=60020: caught a
> ClosedChannelException, this means that the server /0.0.0.0:60020 was
> processing a request but the client went away. The error message was: null
> 2017-01-16 17:17:00,518 WARN org.apache.hadoop.hbase.util.Sleeper: We slept
> 84292ms instead of 3000ms, this is likely due to a long garbage collecting
> pause and it's usually bad, see
> http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
> 2017-01-16 17:17:00,517 WARN org.apache.hadoop.hdfs.DFSClient:
> DFSOutputStream ResponseProcessor exception for block
> BP-730307917-10.93.1.221-1475979381504:blk_1079410730_7411916
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2241)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:235)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:971)
> 2017-01-16 17:17:00,517 INFO org.apache.zookeeper.ClientCnxn: Client session
> timed out, have not heard from server in 98152ms for sessionid
> 0x458f196c5cbdbb0, closing socket connection and attempting reconnect
> 2017-01-16 17:17:00,586 WARN org.apache.hadoo
[jira] [Updated] (HBASE-17476) FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server host04.byzoro.com,60020,1484273922270: org.apache.hadoop.hbase.YouAreDeadException: Ser
[
https://issues.apache.org/jira/browse/HBASE-17476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17476:
-
Description:
{noformat}
GC pool 'ParNew' had collection(s): count=1 time=3237ms
2017-01-16 17:15:25,185 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor:
Detected pause in JVM or host machine (eg GC): pause of approximately 1319ms
GC pool 'ParNew' had collection(s): count=1 time=1697ms
2017-01-16 17:15:25,215 WARN org.apache.hadoop.hbase.ipc.RpcServer:
(responseTooSlow):
{"processingtimems":13043,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:59138","starttimems":1484558112172,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
2017-01-16 17:15:25,606 WARN org.apache.hadoop.hbase.ipc.RpcServer:
(responseTooSlow):
{"processingtimems":16061,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:59114","starttimems":1484558109545,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
2017-01-16 17:15:25,901 WARN org.apache.hadoop.hbase.ipc.RpcServer:
(responseTooSlow):
{"processingtimems":11495,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:60177","starttimems":1484558114405,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
2017-01-16 17:15:25,991 WARN org.apache.hadoop.hbase.ipc.RpcServer:
(responseTooSlow):
{"processingtimems":12069,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.224:59673","starttimems":1484558113922,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
2017-01-16 17:15:27,000 WARN org.apache.hadoop.hbase.ipc.RpcServer:
(responseTooSlow):
{"processingtimems":15320,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.226:39885","starttimems":1484558111679,"queuetimems":0,"class":"HRegionServer","responsesize":14,"method":"Scan"}
2017-01-16 17:15:30,032 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor:
Detected pause in JVM or host machine (eg GC): pause of approximately 2846ms
GC pool 'ParNew' had collection(s): count=1 time=3025ms
2017-01-16 17:17:00,517 INFO org.apache.zookeeper.ClientCnxn: Client session
timed out, have not heard from server in 98152ms for sessionid
0x458f196c5cbdbaf, closing socket connection and attempting reconnect
2017-01-16 17:17:00,517 INFO org.apache.zookeeper.ClientCnxn: Client session
timed out, have not heard from server in 98152ms for sessionid
0x458f196c5cbdbad, closing socket connection and attempting reconnect
2017-01-16 17:17:00,517 WARN org.apache.hadoop.hbase.ipc.RpcServer:
PriorityRpcServer.handler=3,queue=1,port=60020: caught a
ClosedChannelException, this means that the server /0.0.0.0:60020 was
processing a request but the client went away. The error message was: null
2017-01-16 17:17:00,518 WARN org.apache.hadoop.hbase.util.Sleeper: We slept
84292ms instead of 3000ms, this is likely due to a long garbage collecting
pause and it's usually bad, see
http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
2017-01-16 17:17:00,517 WARN org.apache.hadoop.hdfs.DFSClient: DFSOutputStream
ResponseProcessor exception for block
BP-730307917-10.93.1.221-1475979381504:blk_1079410730_7411916
java.io.EOFException: Premature EOF: no length prefix available
at
org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2241)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:235)
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:971)
2017-01-16 17:17:00,517 INFO org.apache.zookeeper.ClientCnxn: Client session
timed out, have not heard from server in 98152ms for sessionid
0x458f196c5cbdbb0, closing socket connection and attempting reconnect
2017-01-16 17:17:00,586 WARN org.apache.hadoop.hbase.ipc.RpcServer:
(responseTooSlow):
{"processingtimems":84009,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.225:57992","starttimems":1484558136577,"queuetimems":0,"class":"HRegionServer","responsesize":12,"method":"Scan"}
2017-01-16 17:17:00,580 WARN org.apache.hadoop.hbase.ipc.RpcServer:
(responseTooSlow):
{"processingtimems":82792,"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","client":"193.168.1.223:50898","starttimems":1484558137788,"queuetimems":0,"class":"HRegionServer","responsesize":12,"method":"Scan"}
2017-01-16 17:17:00,518 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionChecker: Chore:
CompactionChecker missed its start time
2017-01-16 17:17:00,518 INFO
org.apache.hadoop.hbase.regionserver.HeapMemoryManager$HeapMemoryTunerChore:
Chore: host04.byzoro.com,6002
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826565#comment-15826565 ] ramkrishna.s.vasudevan commented on HBASE-17081: One minor comment, in CompositeImmutableSegment see if we can unify getSnapshotScanner and getScanner by passing the corresponding readPt and order. Because both gets created on the list of segments only? [~ted_yu], [~saint@gmail.com] any other comments before we commit this? > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0 > > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBASE-17081-V07.patch, HBASE-17081-V10.patch, HBASE-17081-V13.patch, > HBASE-17081-V14.patch, HBASE-17081-V15.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17437) Support specifying a WAL directory outside of the root directory
[ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826568#comment-15826568 ] Sean Busbey commented on HBASE-17437: - are review board and master.004 the same? if not, which should I focus on? > Support specifying a WAL directory outside of the root directory > > > Key: HBASE-17437 > URL: https://issues.apache.org/jira/browse/HBASE-17437 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration, wal >Affects Versions: 1.2.4 >Reporter: Yishan Yang >Assignee: Zach York > Labels: patch > Fix For: 2.0.0, 1.4.0 > > Attachments: hbase-17437-branch-1.2.patch, > HBASE-17437.master.001.patch, HBASE-17437.master.002.patch, > HBASE-17437.master.003.patch, HBASE-17437.master.004.patch, > hbase-17437-master.patch > > > Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some > FileSystems (such as Amazon S3) don’t support append or consistent writes. > These two properties are imperative for the WAL in order to avoid loss of > writes. However, StoreFiles don’t necessarily need the same consistency > guarantees (since writes are cached locally and if writes fail, they can > always be replayed from the WAL). > > This JIRA aims to allow users to configure a log directory (for WALs) that is > outside of the root directory or even in a different FileSystem. The default > value will still put the log directory under the root directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826492#comment-15826492 ] Manjunath Anand commented on HBASE-17469: - Thanks a lot [~yuzhih...@gmail.com] for your help and quick response. This is my first assignment and I am delighted :-) > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: Ted Yu >Assignee: Manjunath Anand > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17469: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the patch, Manjunath > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: Ted Yu >Assignee: Manjunath Anand > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826457#comment-15826457 ] Ted Yu commented on HBASE-17469: Test failure was not related. > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: Ted Yu >Assignee: Manjunath Anand > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[
https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826443#comment-15826443
]
Hadoop QA commented on HBASE-17469:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 26s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 37s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 120m 33s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847826/HBASE-17469.master.002.patch
|
| JIRA Issue | HBASE-17469 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f2f39c2572b4 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9b38c1a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5296/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5296/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: HBase-16261-V8.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase, mapreduce >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, > HBase-16261-V6.patch, HBase-16261-V7.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: (was: HBase-16261-V8.patch) > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase, mapreduce >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, > HBase-16261-V6.patch, HBase-16261-V7.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17445) Count size of serialized exceptions in checking max result size quota
[ https://issues.apache.org/jira/browse/HBASE-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17445: --- Fix Version/s: 1.3.1 > Count size of serialized exceptions in checking max result size quota > - > > Key: HBASE-17445 > URL: https://issues.apache.org/jira/browse/HBASE-17445 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0, 1.3.1 > > Attachments: 17445.branch-1.v2.txt, 17445.v1.txt, 17445.v2.txt > > > Currently RSRpcServices#doNonAtomicRegionMutation() does accounting for cell > size and block size against the max result size quota. > However, as HBASE-17387 and related JIRAs have shown, we should consider the > size of exceptions sent back to the client as well. > This issue adds accounting for size of serialized exceptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17387) Reduce the overhead of exception report in RegionActionResult for multi()
[
https://issues.apache.org/jira/browse/HBASE-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-17387:
---
Fix Version/s: 1.3.1
> Reduce the overhead of exception report in RegionActionResult for multi()
> -
>
> Key: HBASE-17387
> URL: https://issues.apache.org/jira/browse/HBASE-17387
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
> Attachments: 17387.v1.txt, 17387.v2.txt, 17387.v3.txt
>
>
> For RSRpcServices#doNonAtomicRegionMutation() :
> {code}
> for (ClientProtos.Action action: actions.getActionList()) {
> ...
> } catch (IOException ie) {
> rpcServer.getMetrics().exception(ie);
> resultOrExceptionBuilder = ResultOrException.newBuilder().
> setException(ResponseConverter.buildException(ie));
> }
> if (resultOrExceptionBuilder != null) {
> // Propagate index.
> resultOrExceptionBuilder.setIndex(action.getIndex());
> builder.addResultOrException(resultOrExceptionBuilder.build());
> }
> {code}
> The exceptions are added to builder in the for loop.
> The ClientProtos.ResultOrException.Builder instance is created within the for
> loop.
> For large multi call, this may incur non-trivial overhead for garbage
> collector if there're many exceptions.
> e.g. Here was sample debug log showing the actions in a batch:
> {code}
> 2016-12-23 04:21:56,263 DEBUG
> org.apache.hadoop.hbase.regionserver.RSRpcServices: NonAtomicRegionMutation
> batch summary: numAppends=0, numDeletes=11, numGets=0, numIncrements=15638,
> numPuts=15627, numServiceCalls=0, serializedSize=3871713,
> user=hbase/pob3.G.COM (auth:KERBEROS), client=null
> {code}
> Note the large number of increments in the batch.
> When the increments encounter conflict at server side, the overhead of
> stringified exceptions in response is considerable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15347) Update CHANGES.txt for 1.3
[ https://issues.apache.org/jira/browse/HBASE-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826187#comment-15826187 ] Mikhail Antonov commented on HBASE-15347: - Yeah that's right. I think last I checked this question (few months ago) we didn't have consistent practice in place to handle deviating site docs. > Update CHANGES.txt for 1.3 > -- > > Key: HBASE-15347 > URL: https://issues.apache.org/jira/browse/HBASE-15347 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Fix For: 1.3.0 > > Attachments: HBASE-15347-branch-1.3.v1.patch, HBASE-15347.patch, > HBASE-15347-v2.branch-1.3.patch > > > Going to post the steps in preparing changes file for 1.3 here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath Anand updated HBASE-17469: Component/s: findbugs > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: Ted Yu >Assignee: Manjunath Anand > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17470) Remove merge region code from region server
[ https://issues.apache.org/jira/browse/HBASE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826174#comment-15826174 ] stack commented on HBASE-17470: --- It is ok to remove dispatchMergingRegions w/o deprecation? DId that come in to master branch only? Or is that the incompatible change we are talking about here? In here we are removing MergeRegionsRequest. Were these added by HBASE-16119 ? i.e. only in master branch so safe to remove them? We deprecate CP methods. There are others to use in stead? In the deprecate message you want to point folks at what to use in their place? Ditto on what to use in stead of dispatchMergingRegions? Is that right in HMaster? YOu remove dispatchMergingRegions... Does that make the deprecations noops? Otherwise patch looks good. Needs release note pointing at justification for the incompatible change and what users need to use instead doing merges. Is there any test of region merge in code base now given how much is removed here? Looks good [~syuanjiang] > Remove merge region code from region server > --- > > Key: HBASE-17470 > URL: https://issues.apache.org/jira/browse/HBASE-17470 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0 > > Attachments: HBASE-17470.v1-master.patch, HBASE-17470.v2-master.patch > > > HBASE-16119 moves the merge region to the master-side. There is no need to > keep region_server-side merge region code to remove logic duplication. > util.Merge and HMerge tools depends on RS-side merge region logic. However, > now we can merge regions using shell command. It is dangerous to do offline > merge. For 2.0, it is a good time to remove those out-of-date tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-17279) TestSerialReplication#testRegionSplit is flaky in branch-1
[
https://issues.apache.org/jira/browse/HBASE-17279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-17279.
Resolution: Cannot Reproduce
> TestSerialReplication#testRegionSplit is flaky in branch-1
> --
>
> Key: HBASE-17279
> URL: https://issues.apache.org/jira/browse/HBASE-17279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Phil Yang
>Priority: Minor
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/4832/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> testRegionSplit(org.apache.hadoop.hbase.replication.TestSerialReplication)
> Time elapsed: 24.247 sec <<< FAILURE!
> java.lang.AssertionError: expected:<9> but was:<4>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:631)
> at
> org.apache.hadoop.hbase.replication.TestSerialReplication.testRegionSplit(TestSerialReplication.java:256)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826162#comment-15826162 ] Anoop Sam John commented on HBASE-17081: +1 > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0 > > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBASE-17081-V07.patch, HBASE-17081-V10.patch, HBASE-17081-V13.patch, > HBASE-17081-V14.patch, HBASE-17081-V15.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath Anand updated HBASE-17469: Fix Version/s: 1.4.0 2.0.0 Status: Patch Available (was: In Progress) > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Manjunath Anand > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826160#comment-15826160 ] Ted Yu commented on HBASE-17469: lgtm, pending QA. > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Manjunath Anand > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826158#comment-15826158 ] stack commented on HBASE-16630: --- bq. The patch is fine and would be harmless. If its harmless, why commit it (smile)? You think the patch will help with fragmentation? There was to be a unit test added [~dvdreddy]? Too hard to add? Thanks. > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630.patch, HBASE-16630-v2.patch, HBASE-16630-v3.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826154#comment-15826154 ] Manjunath Anand commented on HBASE-17469: - Updated the description to included the changes incorporated and came up with 002 patch. [~yuzhih...@gmail.com] please review and let me know your thoughts. Meanwhile I will submit the patch for a QA run. Thanks > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Manjunath Anand > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17045) Unify the implementation of small scan and regular scan
[ https://issues.apache.org/jira/browse/HBASE-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826152#comment-15826152 ] Duo Zhang commented on HBASE-17045: --- The failed UT is related. Let me dig more. > Unify the implementation of small scan and regular scan > --- > > Key: HBASE-17045 > URL: https://issues.apache.org/jira/browse/HBASE-17045 > Project: HBase > Issue Type: Sub-task > Components: Client, scan >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17045.patch, HBASE-17045-v1.patch, > HBASE-17045-v2.patch, HBASE-17045-v3.patch > > > See [~enis]'s comment in HBASE-16838 > https://issues.apache.org/jira/browse/HBASE-16838?focusedCommentId=15637803&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15637803 > But there is another scenario that we need small scan is that, we do not know > the stop row but we only want a small set of results. For example, in the > implementation of region locator, we will use small scan and set caching to 1 > as we only need one row. > So I think we need to add a new option(maybe called limit?) for the scan > object, and deprecate the small option. And the server side modification > should also be committed to branch-1 to simplify the logic of async client in > 2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath Anand updated HBASE-17469: Description: HBASE-17450 handles the empty table name in equals(). This JIRA is to properly handle empty TableName in TablePermission#readFields() and TablePermission#write() methods and also null checks whereever appropriate in the code to avoid NPE. The toString method if-elseif-else condition check causes some of the details not to be logged for a given TablePermission so this has to be fixed was: HBASE-17450 handles the empty table name in equals(). This JIRA is to properly handle empty TableName in TablePermission#readFields() and TablePermission#write() methods. > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Manjunath Anand > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods and also > null checks whereever appropriate in the code to avoid NPE. The toString > method if-elseif-else condition check causes some of the details not to be > logged for a given TablePermission so this has to be fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17477) Set capacity on ArrayList in hbase-common
[
https://issues.apache.org/jira/browse/HBASE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826151#comment-15826151
]
Hadoop QA commented on HBASE-17477:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 1s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 45s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.2 Server=1.12.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847819/HBASE-17477.master.001.patch
|
| JIRA Issue | HBASE-17477 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 3d499ab793f3 3.13.0-100-generic #147-Ubuntu SMP Tue Oct 18
16:48:51 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9b38c1a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5295/testReport/ |
| modules | C: hbase-common U: hbase-common |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5295/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Set capacity on ArrayList in hbase-common
> -
>
> Key: HBASE-17477
> URL: https://issues.apache.org/jira/browse/HBASE-17477
> Project: HBase
> Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Attachments: HBASE-17477.master.001.patch
>
>
> Set the capacity on an ArrayList when possible in
[jira] [Updated] (HBASE-17469) Properly handle empty TableName in TablePermission#readFields and #write
[ https://issues.apache.org/jira/browse/HBASE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath Anand updated HBASE-17469: Attachment: HBASE-17469.master.002.patch > Properly handle empty TableName in TablePermission#readFields and #write > > > Key: HBASE-17469 > URL: https://issues.apache.org/jira/browse/HBASE-17469 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Manjunath Anand > Attachments: HBASE-17469.master.001.patch, > HBASE-17469.master.002.patch > > > HBASE-17450 handles the empty table name in equals(). > This JIRA is to properly handle empty TableName in > TablePermission#readFields() and TablePermission#write() methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17361) Make HTable thread safe
[
https://issues.apache.org/jira/browse/HBASE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826121#comment-15826121
]
Yu Li commented on HBASE-17361:
---
Ok, then let me prepare a patch with the TableBuilder way. Thanks [~Apache9]
and [~stack] for the suggestion/confirmation.
> Make HTable thread safe
> ---
>
> Key: HBASE-17361
> URL: https://issues.apache.org/jira/browse/HBASE-17361
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Attachments: HBASE-17361.patch, HBASE-17361.patch
>
>
> Currently HTable is marked as NOT thread safe, and this JIRA target at
> improving this to take better usage of the thread-safe BufferedMutator.
> Some findings/work done:
> If we try to do put to the same HTable instance in parallel, there'll be
> problem, since now we have {{HTable#getBufferedMutator}} like
> {code}
>BufferedMutator getBufferedMutator() throws IOException {
> if (mutator == null) {
> this.mutator = (BufferedMutatorImpl) connection.getBufferedMutator(
> new BufferedMutatorParams(tableName)
> .pool(pool)
> .writeBufferSize(connConfiguration.getWriteBufferSize())
> .maxKeyValueSize(connConfiguration.getMaxKeyValueSize())
> );
> }
> mutator.setRpcTimeout(writeRpcTimeout);
> mutator.setOperationTimeout(operationTimeout);
> return mutator;
> }
> {code}
> And {{HTable#flushCommits}}:
> {code}
> void flushCommits() throws IOException {
> if (mutator == null) {
> // nothing to flush if there's no mutator; don't bother creating one.
> return;
> }
> getBufferedMutator().flush();
> }
> {code}
> For {{HTable#put}}
> {code}
> public void put(final Put put) throws IOException {
> getBufferedMutator().mutate(put);
> flushCommits();
> }
> {code}
> If we launch multiple threads to put in parallel, below sequence might happen
> because {{HTable#getBufferedMutator}} is not thread safe:
> {noformat}
> 1. ThreadA runs to getBufferedMutator and finds mutator==null
> 2. ThreadB runs to getBufferedMutator and finds mutator==null
> 3. ThreadA initialize mutator to instanceA, then calls mutator#mutate,
> adding one put (putA) into {{writeAsyncBuffer}}
> 4. ThreadB initialize mutator to instanceB
> 5. ThreadA runs to flushCommits, now mutator is instanceB, it calls
> instanceB's flush method, putA is lost
> {noformat}
> After fixing this, we will find quite some contention on
> {{BufferedMutatorImpl#flush}}, so more efforts required to make HTable thread
> safe but with good performance meanwhile.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17361) Make HTable thread safe
[
https://issues.apache.org/jira/browse/HBASE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826114#comment-15826114
]
stack commented on HBASE-17361:
---
bq. Is this acceptable?
IMO, yes. I like this suggestion of deprecating first.
> Make HTable thread safe
> ---
>
> Key: HBASE-17361
> URL: https://issues.apache.org/jira/browse/HBASE-17361
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Attachments: HBASE-17361.patch, HBASE-17361.patch
>
>
> Currently HTable is marked as NOT thread safe, and this JIRA target at
> improving this to take better usage of the thread-safe BufferedMutator.
> Some findings/work done:
> If we try to do put to the same HTable instance in parallel, there'll be
> problem, since now we have {{HTable#getBufferedMutator}} like
> {code}
>BufferedMutator getBufferedMutator() throws IOException {
> if (mutator == null) {
> this.mutator = (BufferedMutatorImpl) connection.getBufferedMutator(
> new BufferedMutatorParams(tableName)
> .pool(pool)
> .writeBufferSize(connConfiguration.getWriteBufferSize())
> .maxKeyValueSize(connConfiguration.getMaxKeyValueSize())
> );
> }
> mutator.setRpcTimeout(writeRpcTimeout);
> mutator.setOperationTimeout(operationTimeout);
> return mutator;
> }
> {code}
> And {{HTable#flushCommits}}:
> {code}
> void flushCommits() throws IOException {
> if (mutator == null) {
> // nothing to flush if there's no mutator; don't bother creating one.
> return;
> }
> getBufferedMutator().flush();
> }
> {code}
> For {{HTable#put}}
> {code}
> public void put(final Put put) throws IOException {
> getBufferedMutator().mutate(put);
> flushCommits();
> }
> {code}
> If we launch multiple threads to put in parallel, below sequence might happen
> because {{HTable#getBufferedMutator}} is not thread safe:
> {noformat}
> 1. ThreadA runs to getBufferedMutator and finds mutator==null
> 2. ThreadB runs to getBufferedMutator and finds mutator==null
> 3. ThreadA initialize mutator to instanceA, then calls mutator#mutate,
> adding one put (putA) into {{writeAsyncBuffer}}
> 4. ThreadB initialize mutator to instanceB
> 5. ThreadA runs to flushCommits, now mutator is instanceB, it calls
> instanceB's flush method, putA is lost
> {noformat}
> After fixing this, we will find quite some contention on
> {{BufferedMutatorImpl#flush}}, so more efforts required to make HTable thread
> safe but with good performance meanwhile.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
