[jira] [Commented] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359881#comment-16359881
 ] 

Hadoop QA commented on HBASE-19844:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red}  0m 
23s{color} | {color:red} The patch generated 7 new + 383 unchanged - 2 fixed = 
390 total (was 385) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red}  0m  
3s{color} | {color:red} The patch generated 4 new + 729 unchanged - 3 fixed = 
733 total (was 732) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  7m 
21s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
10s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m  2s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19844 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12910084/HBASE-19844.master.001.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  rubocop  ruby_lint  |
| uname | Linux e6ab072452de 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 0593dda663 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| rubocop | v0.52.1 |
| rubocop | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11483/artifact/patchprocess/diff-patch-rubocop.txt
 |
| ruby-lint | v2.3.1 |
| ruby-lint | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11483/artifact/patchprocess/diff-patch-ruby-lint.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11483/testReport/ |
| Max. process+thread count | 1864 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11483/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Eshcar Hillel (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eshcar Hillel updated HBASE-18294:
--
Attachment: HBASE-18294.01.patch

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, 
> HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, 
> HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, 
> HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, 
> HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, 
> HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.master.01.patch, 
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Duo Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19082:
--
Labels:   (was: HBASE-19064)

> Reject read/write from client but accept write from replication in state S
> --
>
> Key: HBASE-19082
> URL: https://issues.apache.org/jira/browse/HBASE-19082
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-19082-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Duo Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19082:
--
Attachment: HBASE-19082-HBASE-19064.patch

> Reject read/write from client but accept write from replication in state S
> --
>
> Key: HBASE-19082
> URL: https://issues.apache.org/jira/browse/HBASE-19082
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Priority: Major
>  Labels: HBASE-19064
> Attachments: HBASE-19082-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Duo Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19082:
--
Status: Patch Available  (was: Open)

> Reject read/write from client but accept write from replication in state S
> --
>
> Key: HBASE-19082
> URL: https://issues.apache.org/jira/browse/HBASE-19082
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-19082-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Duo Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reassigned HBASE-19082:
-

 Assignee: Duo Zhang
Fix Version/s: (was: 3.0.0)

> Reject read/write from client but accept write from replication in state S
> --
>
> Key: HBASE-19082
> URL: https://issues.apache.org/jira/browse/HBASE-19082
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>  Labels: HBASE-19064
> Attachments: HBASE-19082-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19959) How much RAM space is to be really consumed by the memstore?

2018-02-11 Thread Chance Li (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359868#comment-16359868
 ] 

Chance Li commented on HBASE-19959:
---

To flush all the data to make the memstore size is 0 . 

> How much RAM space is to be really consumed by the memstore?
> 
>
> Key: HBASE-19959
> URL: https://issues.apache.org/jira/browse/HBASE-19959
> Project: HBase
>  Issue Type: Brainstorming
>  Components: regionserver
>Reporter: Chance Li
>Priority: Minor
>
> Let's consider this scenario where memstoreLAB and ChunkPool is enable and 
> max memstore size is 10G, and after some time all pooled chunk have been 
> created, then flush all data, now memstore size is 0 but RAM actually have 
> consumed 10G, then continue writing big cell which will not use the chunk 
> pool but jvm heap, then memstore size will be increased to 10G(maybe more 
> because overhead). now we can see RAM actually consumed 20G (10G of pooled 
> chunk + 10G java objects), but the max memstore size is only 10G.
> what I say is the max memstore size not only take care about the cell "size" 
> but also RAM really used. This will be a strict memory management: the max 
> memstore size limit the RAM space which the memstore or related module can be 
> used.
> This really rarely occured. It's just for a robust memory managemant 
> semantically. 
>  What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Eshcar Hillel (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eshcar Hillel updated HBASE-18294:
--
Attachment: HBASE-18294.01.patch

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.02.patch, HBASE-18294.03.patch, HBASE-18294.04.patch, 
> HBASE-18294.05.patch, HBASE-18294.06.patch, HBASE-18294.07.patch, 
> HBASE-18294.07.patch, HBASE-18294.08.patch, HBASE-18294.09.patch, 
> HBASE-18294.10.patch, HBASE-18294.11.patch, HBASE-18294.11.patch, 
> HBASE-18294.12.patch, HBASE-18294.13.patch, HBASE-18294.15.patch, 
> HBASE-18294.16.patch, HBASE-18294.master.01.patch, HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Zheng Hu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-19972:
-
Status: Patch Available  (was: Open)

> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> --
>
> Key: HBASE-19972
> URL: https://issues.apache.org/jira/browse/HBASE-19972
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19972-branch-1.4.patch, HBASE-19972.v1.patch
>
>
> As [~Apache9] said in HBASE-12091. 
> In ReplicationSink#batch,we swallow the RetriesExhaustedWithDetailsException 
> except 
> TableNotFoundException,   actually,  should rethrow the exception. 
> {code:java}
> try {
>   Connection connection = getConnection();
>   table = connection.getTable(tableName);
>   for (List rows : allRows) {
> table.batch(rows);
>   }
> } catch (RetriesExhaustedWithDetailsException rewde) {
>   for (Throwable ex : rewde.getCauses()) {
> if (ex instanceof TableNotFoundException) {
>   throw new TableNotFoundException("'"+tableName+"'");
> }
>   }
> } 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359906#comment-16359906
 ] 

Ted Yu commented on HBASE-19844:


Can you look at "flush should work" in 
hbase-shell/src/test//ruby/hbase/admin_test.rb and add a similar test ?

> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch, 
> HBASE-19844.master.002.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359910#comment-16359910
 ] 

Hadoop QA commented on HBASE-19972:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
47s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
35s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
37s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
19s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
56s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
12s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
26s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  2m 
36s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
8m 48s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1 
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 23s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 50s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.util.TestHBaseFsck |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:74e3133 |
| JIRA Issue | HBASE-19972 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12910075/HBASE-19972-branch-1.4.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  

[jira] [Updated] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Reid Chan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-19844:
--
Attachment: HBASE-19844.master.001.patch

> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Eshcar Hillel (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eshcar Hillel updated HBASE-18294:
--
Attachment: HBASE-18294.master.01.patch

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, 
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, 
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, 
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, 
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, 
> HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, 
> HBASE-18294.master.01.patch, HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Reid Chan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-19844:
--
Status: Patch Available  (was: Open)

Already checked shell command on local machine.

> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Eshcar Hillel (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eshcar Hillel updated HBASE-18294:
--
Status: Patch Available  (was: Open)

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.02.patch, HBASE-18294.03.patch, HBASE-18294.04.patch, 
> HBASE-18294.05.patch, HBASE-18294.06.patch, HBASE-18294.07.patch, 
> HBASE-18294.07.patch, HBASE-18294.08.patch, HBASE-18294.09.patch, 
> HBASE-18294.10.patch, HBASE-18294.11.patch, HBASE-18294.11.patch, 
> HBASE-18294.12.patch, HBASE-18294.13.patch, HBASE-18294.15.patch, 
> HBASE-18294.16.patch, HBASE-18294.master.01.patch, HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Reid Chan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359890#comment-16359890
 ] 

Reid Chan commented on HBASE-19844:
---

Try to fix {{rubocop}} and {{ruby-lint}}

> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch, 
> HBASE-19844.master.002.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Reid Chan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-19844:
--
Attachment: HBASE-19844.master.002.patch

> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch, 
> HBASE-19844.master.002.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359907#comment-16359907
 ] 

Hadoop QA commented on HBASE-18294:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 17 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  7m 
15s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m 
17s{color} | {color:red} hbase-server: The patch generated 1 new + 806 
unchanged - 7 fixed = 807 total (was 813) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
41s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
18m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
16s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
55s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 59s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m 
57s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
51s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 81m 45s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.TestHeapSize |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18294 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12910086/HBASE-18294.01.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 9a0ed54552aa 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | 

[jira] [Commented] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Reid Chan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359908#comment-16359908
 ] 

Reid Chan commented on HBASE-19844:
---

Thanks for reminder Ted.

> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch, 
> HBASE-19844.master.002.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19959) How much RAM space is to be really consumed by the memstore?

2018-02-11 Thread Anastasia Braginsky (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359851#comment-16359851
 ] 

Anastasia Braginsky commented on HBASE-19959:
-

OK, I may be understood it wrongly what you write. But then:
{quote}Let's consider this scenario where memstoreLAB and ChunkPool is enable 
and max memstore size is 10G, and after some time all pooled chunk have been 
created, then flush all data, now memstore size is 0 
{quote}
Why do you say memstore size is 0? If you consume pool chunks and write there 
data, memstore must have value...

> How much RAM space is to be really consumed by the memstore?
> 
>
> Key: HBASE-19959
> URL: https://issues.apache.org/jira/browse/HBASE-19959
> Project: HBase
>  Issue Type: Brainstorming
>  Components: regionserver
>Reporter: Chance Li
>Priority: Minor
>
> Let's consider this scenario where memstoreLAB and ChunkPool is enable and 
> max memstore size is 10G, and after some time all pooled chunk have been 
> created, then flush all data, now memstore size is 0 but RAM actually have 
> consumed 10G, then continue writing big cell which will not use the chunk 
> pool but jvm heap, then memstore size will be increased to 10G(maybe more 
> because overhead). now we can see RAM actually consumed 20G (10G of pooled 
> chunk + 10G java objects), but the max memstore size is only 10G.
> what I say is the max memstore size not only take care about the cell "size" 
> but also RAM really used. This will be a strict memory management: the max 
> memstore size limit the RAM space which the memstore or related module can be 
> used.
> This really rarely occured. It's just for a robust memory managemant 
> semantically. 
>  What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19844) Shell should support to flush by regionserver

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359900#comment-16359900
 ] 

Hadoop QA commented on HBASE-19844:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red}  0m 
26s{color} | {color:red} The patch generated 4 new + 383 unchanged - 2 fixed = 
387 total (was 385) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red}  0m  
4s{color} | {color:red} The patch generated 3 new + 730 unchanged - 2 fixed = 
733 total (was 732) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  7m 
28s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
10s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 32s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19844 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12910088/HBASE-19844.master.002.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  rubocop  ruby_lint  |
| uname | Linux fe7b1cb17bb3 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 0593dda663 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| rubocop | v0.52.1 |
| rubocop | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11484/artifact/patchprocess/diff-patch-rubocop.txt
 |
| ruby-lint | v2.3.1 |
| ruby-lint | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11484/artifact/patchprocess/diff-patch-ruby-lint.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11484/testReport/ |
| Max. process+thread count | 1846 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11484/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Shell should support to flush by regionserver
> -
>
> Key: HBASE-19844
> URL: https://issues.apache.org/jira/browse/HBASE-19844
> Project: HBase
>  Issue Type: New Feature
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-19844.master.001.patch, 
> HBASE-19844.master.002.patch
>
>
> HBASE-4224 add a method to admin that can do the flush by regionserver. As 
> with other Admin methods, we should enable shell to use the flush method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Duo Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359951#comment-16359951
 ] 

Duo Zhang commented on HBASE-19082:
---

Review board link:

https://reviews.apache.org/r/65597/

> Reject read/write from client but accept write from replication in state S
> --
>
> Key: HBASE-19082
> URL: https://issues.apache.org/jira/browse/HBASE-19082
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-19082-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359980#comment-16359980
 ] 

Hadoop QA commented on HBASE-19082:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
31s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
42s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
56s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
35s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
9s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  1m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
10s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
23s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch hbase-replication passed checkstyle 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
12s{color} | {color:green} hbase-server: The patch generated 0 new + 478 
unchanged - 5 fixed = 478 total (was 483) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
47s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
18m 27s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}  
1m 24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
30s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
18s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
19s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}103m 57s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense 

[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359979#comment-16359979
 ] 

Hadoop QA commented on HBASE-18294:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  2m 
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 17 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
 8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  8m 
 7s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
34s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
35s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
25s{color} | {color:green} hbase-server: The patch generated 0 new + 806 
unchanged - 7 fixed = 806 total (was 813) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
22s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
34s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
21m 36s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
29s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m  
3s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 11s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m 
34s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
18s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 58s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestClientPushback |

[jira] [Commented] (HBASE-19950) Introduce a ColumnValueFilter

2018-02-11 Thread Mike Drob (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360030#comment-16360030
 ] 

Mike Drob commented on HBASE-19950:
---

nit: javadoc on createFilterFromArguments please

> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch, 
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch, 
> HBASE-19950.master.004.patch, HBASE-19950.master.005.patch, 
> HBASE-19950.master.006.patch, HBASE-19950.master.007.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when 
> specified condition is matched, this new filter will return the matched cell 
> only. There're already some discussions in HBASE-19824.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Eshcar Hillel (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eshcar Hillel updated HBASE-18294:
--
Attachment: HBASE-18294.01.patch

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, 
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, 
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, 
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, 
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, 
> HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, 
> HBASE-18294.master.01.patch, HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Eshcar Hillel (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360087#comment-16360087
 ] 

Eshcar Hillel commented on HBASE-18294:
---

TestClientPushback is a flaky test, sometimes passes sometimes fails; I am not 
sure if its failure is related to this patch.

In the test the load of the region is calculated by asking the region for its 
heap size, and there is a comment above the calculation saying 
{color:#808080}"get the current load on RS. Hopefully memstore isn't flushed 
since we wrote the the data"{color} {color:#33}(Hopefully ??!?), then this 
load is compared against the stats returned from the server and if they are not 
equal the test fails. The comment says "{color}{color:#808080}check to see we 
found some load on the memstore"{color}{color:#33}  I guess it was meant to 
be "same load" but in some cases a flush causes the 
difference.{color}{color:#808080}
{color}

{color:#33}It could be that due to this patch the memstore is flushed 
earlier than before and so the test fails more frequently, but it seems to be 
unreasonable to *hope* for no flush during the test.{color}

{color:#33}My solution was simply to double the flush size for this test. 
Any other more elegant suggestions for handling this?{color}

> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, 
> HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, 
> HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, 
> HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, 
> HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, 
> HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, 
> HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.master.01.patch, 
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size 
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the 
> store to another threshold (that can be configured with 
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size 
> (key-value only) to the threshold where it should compare the heap size 
> (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360309#comment-16360309
 ] 

Hadoop QA commented on HBASE-19972:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
45s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
42s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
19m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}100m  
1s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 49s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19972 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12910131/HBASE-19972.v1.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux d49f11863bfc 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 0593dda663 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11489/testReport/ |
| Max. process+thread count | 4852 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/11489/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> 

[jira] [Updated] (HBASE-19950) Introduce a ColumnValueFilter

2018-02-11 Thread Reid Chan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-19950:
--
Attachment: HBASE-19950.master.008.patch

> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch, 
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch, 
> HBASE-19950.master.004.patch, HBASE-19950.master.005.patch, 
> HBASE-19950.master.006.patch, HBASE-19950.master.007.patch, 
> HBASE-19950.master.008.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when 
> specified condition is matched, this new filter will return the matched cell 
> only. There're already some discussions in HBASE-19824.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19950) Introduce a ColumnValueFilter

2018-02-11 Thread Reid Chan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360315#comment-16360315
 ] 

Reid Chan commented on HBASE-19950:
---

v8 patch adds javadoc for method #createFilterFromArguments(ArrayList) and 
#parseFrom(byte[]), and one more test case to address Anoop's concerns. 

Following snippet shows with {{#addColumn(...)}}, results will be limited to 
specified column, that's why i said {{#addColumn(...)}} is a strong constraint. 
While test case 7 without column constraint will scan more results. Both 
behaviors are in line with expectations, and i think this filter should read 
other CFs by default.
Hoping that i make it clear, and address your concern, [~anoop.hbase].
{code}
// 9. Test specified columns with FilterList(MUST_PASS_ONE)
// (ColumnValueFilter, != f[1]:q5[1]:v[1]) || (QualifierFilter, = q5[0])
// Result is different from Test 7, because column is strongly constrained 
by specified columns
Scan anotherScan = new Scan().addColumn(FAMILIES[1], QUALIFIERS_FIVE[1])
  .setFilter(new FilterList(Operator.MUST_PASS_ONE, orFilters));
KeyValue[] expectedValues =
  { new KeyValue(ROWS_TWO[0], FAMILIES[1], QUALIFIERS_FIVE[1], VALUES[0]),
new KeyValue(ROWS_TWO[1], FAMILIES[1], QUALIFIERS_FIVE[1], VALUES[0]),
new KeyValue(ROWS_TWO[2], FAMILIES[1], QUALIFIERS_FIVE[1], VALUES[0]),
new KeyValue(ROWS_TWO[3], FAMILIES[1], QUALIFIERS_FIVE[1], VALUES[0]) };
verifyScanFull(anotherScan, expectedValues);
{code}

BTW, please allow me to break the checkstyle of {{MethodLength}} exception from 
method {{#testColumnValueFilter()}}, becauses test cases should cover enough, 
and not let it block the commitment. :P

> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch, 
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch, 
> HBASE-19950.master.004.patch, HBASE-19950.master.005.patch, 
> HBASE-19950.master.006.patch, HBASE-19950.master.007.patch, 
> HBASE-19950.master.008.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when 
> specified condition is matched, this new filter will return the matched cell 
> only. There're already some discussions in HBASE-19824.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19950) Introduce a ColumnValueFilter

2018-02-11 Thread Reid Chan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-19950:
--
Attachment: (was: HBASE-19950.master.008.patch)

> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch, 
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch, 
> HBASE-19950.master.004.patch, HBASE-19950.master.005.patch, 
> HBASE-19950.master.006.patch, HBASE-19950.master.007.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when 
> specified condition is matched, this new filter will return the matched cell 
> only. There're already some discussions in HBASE-19824.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19950) Introduce a ColumnValueFilter

2018-02-11 Thread Reid Chan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360316#comment-16360316
 ] 

Reid Chan commented on HBASE-19950:
---

I uploaded a wrong v8 patch...

> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch, 
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch, 
> HBASE-19950.master.004.patch, HBASE-19950.master.005.patch, 
> HBASE-19950.master.006.patch, HBASE-19950.master.007.patch, 
> HBASE-19950.master.008.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when 
> specified condition is matched, this new filter will return the matched cell 
> only. There're already some discussions in HBASE-19824.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19950) Introduce a ColumnValueFilter

2018-02-11 Thread Reid Chan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-19950:
--
Attachment: HBASE-19950.master.008.patch

> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch, 
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch, 
> HBASE-19950.master.004.patch, HBASE-19950.master.005.patch, 
> HBASE-19950.master.006.patch, HBASE-19950.master.007.patch, 
> HBASE-19950.master.008.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when 
> specified condition is matched, this new filter will return the matched cell 
> only. There're already some discussions in HBASE-19824.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Duo Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19082:
--
Attachment: HBASE-19082-HBASE-19064-v1.patch

> Reject read/write from client but accept write from replication in state S
> --
>
> Key: HBASE-19082
> URL: https://issues.apache.org/jira/browse/HBASE-19082
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-19082-HBASE-19064-v1.patch, 
> HBASE-19082-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360164#comment-16360164
 ] 

Hadoop QA commented on HBASE-18294:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 17 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  7m 
20s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
30s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
24s{color} | {color:green} hbase-server: The patch generated 0 new + 806 
unchanged - 7 fixed = 806 total (was 813) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
19s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
14s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
21m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
39s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m 
29s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}112m 
19s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m 
29s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 21s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce 

[jira] [Commented] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Duo Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360181#comment-16360181
 ] 

Duo Zhang commented on HBASE-19082:
---

Let me check the failed UTs.

> Reject read/write from client but accept write from replication in state S
> --
>
> Key: HBASE-19082
> URL: https://issues.apache.org/jira/browse/HBASE-19082
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-19082-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19938) Allow write request from replication but reject write request from user client when S state.

2018-02-11 Thread Zheng Hu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360351#comment-16360351
 ] 

Zheng Hu commented on HBASE-19938:
--

This issue is duplicate with HBASE-19082,  and [~Apache9] is working on it, so 
close this issue now. 

> Allow write request from replication but reject write request from user 
> client when S state.
> 
>
> Key: HBASE-19938
> URL: https://issues.apache.org/jira/browse/HBASE-19938
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
>
> According the doc,  we should reject write request  when in S state ,however, 
> the replication data from master cluster will turn to a batch mutation 
> request (it's a write request). 
> So, for peer in S state,  it should  distinguish write request from 
> replication or from user ..  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Duo Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360357#comment-16360357
 ] 

Duo Zhang commented on HBASE-19972:
---

+1.

> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> --
>
> Key: HBASE-19972
> URL: https://issues.apache.org/jira/browse/HBASE-19972
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19972-branch-1.4.patch, HBASE-19972.v1.patch, 
> HBASE-19972.v1.patch
>
>
> As [~Apache9] said in HBASE-12091. 
> In ReplicationSink#batch,we swallow the RetriesExhaustedWithDetailsException 
> except 
> TableNotFoundException,   actually,  should rethrow the exception. 
> {code:java}
> try {
>   Connection connection = getConnection();
>   table = connection.getTable(tableName);
>   for (List rows : allRows) {
> table.batch(rows);
>   }
> } catch (RetriesExhaustedWithDetailsException rewde) {
>   for (Throwable ex : rewde.getCauses()) {
> if (ex instanceof TableNotFoundException) {
>   throw new TableNotFoundException("'"+tableName+"'");
> }
>   }
> } 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Zheng Hu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-19972:
-
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> --
>
> Key: HBASE-19972
> URL: https://issues.apache.org/jira/browse/HBASE-19972
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19972-branch-1.4.patch, HBASE-19972.v1.patch, 
> HBASE-19972.v1.patch
>
>
> As [~Apache9] said in HBASE-12091. 
> In ReplicationSink#batch,we swallow the RetriesExhaustedWithDetailsException 
> except 
> TableNotFoundException,   actually,  should rethrow the exception. 
> {code:java}
> try {
>   Connection connection = getConnection();
>   table = connection.getTable(tableName);
>   for (List rows : allRows) {
> table.batch(rows);
>   }
> } catch (RetriesExhaustedWithDetailsException rewde) {
>   for (Throwable ex : rewde.getCauses()) {
> if (ex instanceof TableNotFoundException) {
>   throw new TableNotFoundException("'"+tableName+"'");
> }
>   }
> } 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Zheng Hu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360373#comment-16360373
 ] 

Zheng Hu commented on HBASE-19972:
--

Pushed to branch-1  & branch-1.4 & branch-2 & master , Thanks [~Apache9] for 
reviewing. 

> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> --
>
> Key: HBASE-19972
> URL: https://issues.apache.org/jira/browse/HBASE-19972
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19972-branch-1.4.patch, HBASE-19972.v1.patch, 
> HBASE-19972.v1.patch
>
>
> As [~Apache9] said in HBASE-12091. 
> In ReplicationSink#batch,we swallow the RetriesExhaustedWithDetailsException 
> except 
> TableNotFoundException,   actually,  should rethrow the exception. 
> {code:java}
> try {
>   Connection connection = getConnection();
>   table = connection.getTable(tableName);
>   for (List rows : allRows) {
> table.batch(rows);
>   }
> } catch (RetriesExhaustedWithDetailsException rewde) {
>   for (Throwable ex : rewde.getCauses()) {
> if (ex instanceof TableNotFoundException) {
>   throw new TableNotFoundException("'"+tableName+"'");
> }
>   }
> } 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19554) AbstractTestDLS.testThreeRSAbort sometimes fails in pre commit

2018-02-11 Thread Duo Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360401#comment-16360401
 ] 

Duo Zhang commented on HBASE-19554:
---

https://builds.apache.org/job/HBASE-Flaky-Tests/25832/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestDLSFSHLog-output.txt/*view*/

{noformat}
2018-02-12 04:56:20,895 DEBUG [PEWorker-16] 
procedure.ServerCrashProcedure(192): pid=132, 
state=RUNNABLE:SERVER_CRASH_PROCESS_META; ServerCrashProcedure 
server=asf911.gq1.ygridcore.net,41715,1518411358686, splitWal=true, meta=true; 
Processing hbase:meta that was on asf911.gq1.ygridcore.net,41715,1518411358686
2018-02-12 04:56:20,895 INFO  [PEWorker-16] procedure2.ProcedureExecutor(1498): 
Initialized subprocedures=[{pid=135, ppid=132, 
state=RUNNABLE:RECOVER_META_SPLIT_LOGS; RecoverMetaProcedure 
failedMetaServer=asf911.gq1.ygridcore.net,41715,1518411358686, splitWal=true}]
{noformat}

Then there is no progress, so at last we time out. Let me add a thread dump 
when we are about to time out to see if we can find something.

> AbstractTestDLS.testThreeRSAbort sometimes fails in pre commit
> --
>
> Key: HBASE-19554
> URL: https://issues.apache.org/jira/browse/HBASE-19554
> Project: HBase
>  Issue Type: Sub-task
>  Components: Recovery, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19554.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/10554/artifact/patchprocess/patch-unit-hbase-server.txt
> The error message is a bit strange:
> {quote}
> [ERROR] testThreeRSAbort(org.apache.hadoop.hbase.master.TestDLSAsyncFSWAL) 
> Time elapsed: 20.627 s <<< ERROR!
> org.apache.hadoop.hbase.TableNotFoundException: Region of 
> 'hbase:namespace,,1513320505933.451650152885a3b41d0b1110deca513c.' is 
> expected in the table of 'testThreeRSAbort', but hbase:meta says it is in the 
> table of 'hbase:namespace'. hbase:meta might be damaged.
> {quote}
> It fails for both FSHLog and AsyncFSWAL. Need to dig more.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Zheng Hu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360332#comment-16360332
 ] 

Zheng Hu commented on HBASE-19972:
--

All UT pass now,   ping [~Apache9] & [~zghaobac] for review ... 

> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> --
>
> Key: HBASE-19972
> URL: https://issues.apache.org/jira/browse/HBASE-19972
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19972-branch-1.4.patch, HBASE-19972.v1.patch, 
> HBASE-19972.v1.patch
>
>
> As [~Apache9] said in HBASE-12091. 
> In ReplicationSink#batch,we swallow the RetriesExhaustedWithDetailsException 
> except 
> TableNotFoundException,   actually,  should rethrow the exception. 
> {code:java}
> try {
>   Connection connection = getConnection();
>   table = connection.getTable(tableName);
>   for (List rows : allRows) {
> table.batch(rows);
>   }
> } catch (RetriesExhaustedWithDetailsException rewde) {
>   for (Throwable ex : rewde.getCauses()) {
> if (ex instanceof TableNotFoundException) {
>   throw new TableNotFoundException("'"+tableName+"'");
> }
>   }
> } 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (HBASE-19938) Allow write request from replication but reject write request from user client when S state.

2018-02-11 Thread Zheng Hu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu resolved HBASE-19938.
--
Resolution: Duplicate

> Allow write request from replication but reject write request from user 
> client when S state.
> 
>
> Key: HBASE-19938
> URL: https://issues.apache.org/jira/browse/HBASE-19938
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
>
> According the doc,  we should reject write request  when in S state ,however, 
> the replication data from master cluster will turn to a batch mutation 
> request (it's a write request). 
> So, for peer in S state,  it should  distinguish write request from 
> replication or from user ..  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Zheng Hu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360220#comment-16360220
 ] 

Zheng Hu commented on HBASE-19972:
--

Re-trigger Hadoop QA for master branch.  The failed TestHBaseFsck ut in 
branch-1 is unrelated to this patch,  If master branch Hadoop QA is OK,  I'll 
commit this patch to master & branch-1 if no objection.

> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> --
>
> Key: HBASE-19972
> URL: https://issues.apache.org/jira/browse/HBASE-19972
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19972-branch-1.4.patch, HBASE-19972.v1.patch, 
> HBASE-19972.v1.patch
>
>
> As [~Apache9] said in HBASE-12091. 
> In ReplicationSink#batch,we swallow the RetriesExhaustedWithDetailsException 
> except 
> TableNotFoundException,   actually,  should rethrow the exception. 
> {code:java}
> try {
>   Connection connection = getConnection();
>   table = connection.getTable(tableName);
>   for (List rows : allRows) {
> table.batch(rows);
>   }
> } catch (RetriesExhaustedWithDetailsException rewde) {
>   for (Throwable ex : rewde.getCauses()) {
> if (ex instanceof TableNotFoundException) {
>   throw new TableNotFoundException("'"+tableName+"'");
> }
>   }
> } 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19326) Remove decommissioned servers from rsgroup

2018-02-11 Thread Xiaolin Ha (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360247#comment-16360247
 ] 

Xiaolin Ha commented on HBASE-19326:


[~andrewcheng][~stack]
In docs it said that 'The default group is not like other rsgroups in that it 
is dynamic. Its server list mirrors the current
state of the cluster; i.e. if you shutdown a server that was part of the 
default rsgroup, and then do a
get_rsgroup default to list its content in the shell, the server will no longer 
be listed. For non-default
groups, though a mode may be offline, it will persist in the non-default 
group’s list of servers. But if
you move the offline server from the non-default rsgroup to default, it will 
not show in the default
list. It will just be dropped.'

But when a regionserver is dead and master is restarted, the regionserver can 
be removed from the non-default rsgroup too.
Is this conflict with the doc? Do you think dead regionservers can be removed 
either?



> Remove decommissioned servers from rsgroup
> --
>
> Key: HBASE-19326
> URL: https://issues.apache.org/jira/browse/HBASE-19326
> Project: HBase
>  Issue Type: New Feature
>  Components: rsgroup
>Affects Versions: 3.0.0, 2.0.0-beta-2
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Fix For: 1.4.0, 2.0.0-beta-1
>
> Attachments: HBASE-19326-branch-1.patch, 
> HBASE-19326.branch-2.001.patch, HBASE-19326.master.001.patch, 
> HBASE-19326.master.002.patch, HBASE-19326.master.003.patch, 
> HBASE-19326.master.004.patch, HBASE-19326.master.005.patch
>
>
> In HBASE-18131, we add an hbase shell command {{clear_deadservers}} to clear 
> deadserver list in ServerManager.
> But rsgroup still contains these dead servers, so we should also remove dead 
> servers from the group information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19972) Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink

2018-02-11 Thread Zheng Hu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-19972:
-
Attachment: HBASE-19972.v1.patch

> Should rethrow  the RetriesExhaustedWithDetailsException when failed to apply 
> the batch in ReplicationSink
> --
>
> Key: HBASE-19972
> URL: https://issues.apache.org/jira/browse/HBASE-19972
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19972-branch-1.4.patch, HBASE-19972.v1.patch, 
> HBASE-19972.v1.patch
>
>
> As [~Apache9] said in HBASE-12091. 
> In ReplicationSink#batch,we swallow the RetriesExhaustedWithDetailsException 
> except 
> TableNotFoundException,   actually,  should rethrow the exception. 
> {code:java}
> try {
>   Connection connection = getConnection();
>   table = connection.getTable(tableName);
>   for (List rows : allRows) {
> table.batch(rows);
>   }
> } catch (RetriesExhaustedWithDetailsException rewde) {
>   for (Throwable ex : rewde.getCauses()) {
> if (ex instanceof TableNotFoundException) {
>   throw new TableNotFoundException("'"+tableName+"'");
> }
>   }
> } 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19082) Reject read/write from client but accept write from replication in state S

2018-02-11 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360243#comment-16360243
 ] 

Hadoop QA commented on HBASE-19082:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
22s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
43s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
55s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
38s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  1m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
 9s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
23s{color} | {color:green} The patch hbase-common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch hbase-replication passed checkstyle 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
12s{color} | {color:green} hbase-server: The patch generated 0 new + 478 
unchanged - 5 fixed = 478 total (was 483) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
47s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
18m 25s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}  
1m 24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
30s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
16s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
19s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}102m 
52s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green}