[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003501#comment-17003501 ] Baiqiang Zhao commented on HBASE-15948: --- Recently, I was reading the source code of branch-1 about RPC, and found that org.apache.hadoop.hbase.ipc.RpcServer.Listener#closeCurrentConnection did not close the connection properly. Master has been fixed in this issue. So can I back this issue to branch-1 sir [~stack] ? > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319924#comment-15319924 ] Hudson commented on HBASE-15948: SUCCESS: Integrated in HBase-Trunk_matrix #1008 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1008/]) Revert "HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns (stack: rev e66ecd7db68d6ef57084543d08f7774c82f22f45) * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SimpleRpcSchedulerFactory.java * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperImpl.java HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns (stack: rev 3a95552cfe6205ae845e1a7e1b5907da55b1a044) * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SimpleRpcSchedulerFactory.java > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319698#comment-15319698 ] stack commented on HBASE-15948: --- Ok. Reverted what was there in case it was wrong version and then applied 004. Thanks for the review [~enis] > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319692#comment-15319692 ] stack commented on HBASE-15948: --- Hmm.. mistaken commit. I'll leave it in but I miscommitted other stuff too... Let me revert that. Thanks for the review [~enis] > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319672#comment-15319672 ] Enis Soztutar commented on HBASE-15948: --- Already committed? Anyway belated +1. > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319658#comment-15319658 ] Hudson commented on HBASE-15948: FAILURE: Integrated in HBase-Trunk_matrix #1007 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1007/]) HBASE-15948 Port "HADOOP-9956 RPC listener inefficiently assigns (stack: rev e0b70c00e74aeaac33570508e3732a53daea839e) * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerWrapperImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SimpleRpcSchedulerFactory.java > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319650#comment-15319650 ] Enis Soztutar commented on HBASE-15948: --- Checking. > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319647#comment-15319647 ] stack commented on HBASE-15948: --- Can I have a +1 on version 004 please? > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[
https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317973#comment-15317973
]
Hadoop QA commented on HBASE-15948:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 47s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m 36s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 27s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808549/HBASE-15948.master.004.patch
|
| JIRA Issue | HBASE-15948 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[
https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315192#comment-15315192
]
Hadoop QA commented on HBASE-15948:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 31s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 23s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 33s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 120m 14s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.procedure.TestProcedureManager |
| | hadoop.hbase.mob.mapreduce.TestMobSweepMapper |
| | hadoop.hbase.master.balancer.TestRegionLocationFinder |
| Timed out junit tests | org.apache.hadoop.hbase.ipc.TestAsyncIPC |
| | org.apache.hadoop.hbase.security.TestAsyncSecureIPC |
| | org.apache.hadoop.hbase.security.TestSecureIPC |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315018#comment-15315018 ] stack commented on HBASE-15948: --- Latest patch incorporates this suggestion. Seems to work. Trying it on hadoopqa. > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.005.patch, Screen Shot > 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314902#comment-15314902 ] stack commented on HBASE-15948: --- [~enis] Radical. I tried it and all broke. Let me looksee... > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[
https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314693#comment-15314693
]
Hadoop QA commented on HBASE-15948:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 13s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 100m 34s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 149m 36s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegion |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808023/HBASE-15948.master.004.patch
|
| JIRA Issue | HBASE-15948 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[
https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313760#comment-15313760
]
Hadoop QA commented on HBASE-15948:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 21s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 13s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 32m 20s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 34s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Unread field:RpcServer.java:[line 616] |
| Failed junit tests | hadoop.hbase.security.TestAsyncSecureIPC |
| | hadoop.hbase.security.TestSecureIPC |
| | hadoop.hbase.master.balancer.TestRegionLocationFinder |
| | hadoop.hbase.ipc.TestIPC |
| Timed out junit tests | org.apache.hadoop.hbase.ipc.TestAsyncIPC |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12807891/HBASE-15948.master.003.patch
|
| JIRA Issue | HBASE-15948 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313634#comment-15313634 ] Enis Soztutar commented on HBASE-15948: --- If channel.register() and selector.select() are thread safe, we don't even need pendingConnections. We can just call register from the acceptor thread. > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, Screen Shot > 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313629#comment-15313629 ] stack commented on HBASE-15948: --- Thanks @enis for review. Are you referring to the register on the channel no longer being done in a synchronized block? That seems to be only place I change concurrency protection (change taken from hadoop). > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, Screen Shot > 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313608#comment-15313608 ] stack commented on HBASE-15948: --- Include Adds HADOOP-9955 RPC idle connection closing is extremely inefficient. In testing, I don't see any throughput benefit but ok to be like upstream hadoop. > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, Screen Shot > 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[
https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313437#comment-15313437
]
Enis Soztutar commented on HBASE-15948:
---
My reading of the Selector javadoc is that Selector.register() and select() are
thread-safe. Is it not the case?
{quote} Selectors are themselves safe for use by multiple concurrent threads;
their key sets, however, are not. {quote}
If not, the patch makes sense.
> Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
> -
>
> Key: HBASE-15948
> URL: https://issues.apache.org/jira/browse/HBASE-15948
> Project: HBase
> Issue Type: Sub-task
> Components: IPC/RPC
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-15948.master.001.patch,
> HBASE-15948.master.002.patch
>
>
> Esteban noticed we were missing this upstream issue. Seems to make no
> difference in profiling but here is the patch anyways.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313404#comment-15313404 ] stack commented on HBASE-15948: --- Hmm.. No difference in throughput in workloadc scenario at least. Slightly less CPU used but that is it. Should probably commit if only to have parity with what is upstream in Hadoop around this bit of the code. > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313402#comment-15313402 ] stack commented on HBASE-15948: --- 002 is after review of current state of hadoop Server... looks like it changed again since HADOOP-9956 using a LinkedBlockingQueue instead of ConcurrentLinkedQueue. Adds new "ipc.server.read.connection-queue.size" config for how many items to queue per Reader instance (this is distinct from "hbase.ipc.server.listen.queue.size" which is how many calls to queue on the socket) > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
