[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219463#comment-15219463 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 48s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 43s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 34s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 59s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 59s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 48s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 48s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 28s {color} | {color:green} hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 5s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_77. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 2s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 23s {color} | {color:red} Patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 62m 35s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_77 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:fbe3e86 | | JIRA Issue | HADOOP-12916 | | GITHUB PR | https://github.com/apache/hadoop/pull/86 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 7e61c97c762d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | |
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218971#comment-15218971 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 11m 47s {color} | {color:red} root in trunk failed. {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 18s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 20s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 42s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 22s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 19s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 34s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 31s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 58s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 29s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m 29s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 59s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 59s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s {color} | {color:green} hadoop-common-project/hadoop-common: patch generated 0 new + 522 unchanged - 45 fixed = 522 total (was 567) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 18s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 22s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_74. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 55s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 31s {color} | {color:red} Patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 101m 3s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_74 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:fbe3e86 | | JIRA Issue | HADOOP-12916 | | GITHUB PR | https://github.com/apache/hadoop/pull/86 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 0d7712a39e4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218505#comment-15218505 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s {color} | {color:red} HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HADOOP-12916 | | GITHUB PR | https://github.com/apache/hadoop/pull/86 | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/8970/console | | Powered by | Apache Yetus 0.2.0 http://yetus.apache.org | This message was automatically generated. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch, > HADOOP-12916.05.patch, HADOOP-12916.06.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218351#comment-15218351 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color} | {color:red} HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HADOOP-12916 | | GITHUB PR | https://github.com/apache/hadoop/pull/86 | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/8968/console | | Powered by | Apache Yetus 0.2.0 http://yetus.apache.org | This message was automatically generated. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch, > HADOOP-12916.05.patch, HADOOP-12916.06.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218317#comment-15218317 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s {color} | {color:red} HADOOP-12916 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HADOOP-12916 | | GITHUB PR | https://github.com/apache/hadoop/pull/86 | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/8967/console | | Powered by | Apache Yetus 0.2.0 http://yetus.apache.org | This message was automatically generated. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch, > HADOOP-12916.05.patch, HADOOP-12916.06.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217939#comment-15217939 ] Tsz Wo Nicholas Sze commented on HADOOP-12916: -- {code} +double decayedAvgRespTime = (responseTimeAvgInLastWindow.get(i) > 0.0) ? +decayFactor * responseTimeAvgInLastWindow.get(i) + +(1 - decayFactor) * averageResponseTime : averageResponseTime; {code} For the decayed case, should the formula be {{decayFactor*responseTimeAvgInLastWindow.get( i) + averageResponseTime}}, i.e. no {{(1 - decayFactor)}} in the second term? BTW, the if-statement can be rewritten as below to make it shorter. {code} final double lastAvg = responseTimeAvgInLastWindow.get(i); if (enableDecay && lastAvg > 0) { final double decayed = decayFactor * lastAvg + averageResponseTime; responseTimeAvgInLastWindow.set(i, decayed); } else { responseTimeAvgInLastWindow.set(i, averageResponseTime); } {code} > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch, > HADOOP-12916.05.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215545#comment-15215545 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 54s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 10s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 48s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 33s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s {color} | {color:green} trunk passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 40s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 10s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 10s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 48s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 48s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s {color} | {color:green} hadoop-common-project/hadoop-common: patch generated 0 new + 522 unchanged - 45 fixed = 522 total (was 567) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 12s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 47s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s {color} | {color:green} the patch passed with JDK v1.8.0_77 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 8s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_77. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 10s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 23s {color} | {color:red} Patch generated 3 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 60m 1s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_77 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:fbe3e86 | | JIRA Issue | HADOOP-12916 | | GITHUB PR | https://github.com/apache/hadoop/pull/86 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 4be9d0f495bb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | |
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214729#comment-15214729 ] ASF GitHub Bot commented on HADOOP-12916: - Github user xiaoyuyao commented on a diff in the pull request: https://github.com/apache/hadoop/pull/86#discussion_r57619429 --- Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java --- @@ -1636,6 +1640,15 @@ public long getTimeDuration(String name, long defaultValue, TimeUnit unit) { return unit.convert(Long.parseLong(vStr), vUnit.unit()); } + public long[] getTimeDurations(String name, TimeUnit unit) { --- End diff -- Good point. I will address that in the next patch. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214591#comment-15214591 ] ASF GitHub Bot commented on HADOOP-12916: - Github user arp7 commented on a diff in the pull request: https://github.com/apache/hadoop/pull/86#discussion_r57606818 --- Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java --- @@ -49,25 +57,68 @@ private final AtomicReferenceputRef; private final AtomicReference takeRef; + private RpcScheduler scheduler; + public CallQueueManager(Class> backingClass, + Class schedulerClass, boolean clientBackOffEnabled, int maxQueueSize, String namespace, Configuration conf) { +int priorityLevels = parseNumLevels(namespace, conf); +this.scheduler = createScheduler(schedulerClass, priorityLevels, +namespace, conf); BlockingQueue bq = createCallQueueInstance(backingClass, - maxQueueSize, namespace, conf); +priorityLevels, maxQueueSize, namespace, conf); this.clientBackOffEnabled = clientBackOffEnabled; this.putRef = new AtomicReference (bq); this.takeRef = new AtomicReference (bq); LOG.info("Using callQueue " + backingClass); } + private static T createScheduler( + Class theClass, int priorityLevels, String ns, Configuration conf) { +// Used for custom, configurable scheduler +try { + Constructor ctor = theClass.getDeclaredConstructor(int.class, + String.class, Configuration.class); + return ctor.newInstance(priorityLevels, ns, conf); +} catch (RuntimeException e) { --- End diff -- See HDFS-9478 which is fixing exception handling when constructing callqueue instances. We could use a similar fix for createScheduler. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214584#comment-15214584 ] ASF GitHub Bot commented on HADOOP-12916: - Github user arp7 commented on a diff in the pull request: https://github.com/apache/hadoop/pull/86#discussion_r57606373 --- Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java --- @@ -1636,6 +1640,15 @@ public long getTimeDuration(String name, long defaultValue, TimeUnit unit) { return unit.convert(Long.parseLong(vStr), vUnit.unit()); } + public long[] getTimeDurations(String name, TimeUnit unit) { --- End diff -- Consider returning a List. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212789#comment-15212789 ] ASF GitHub Bot commented on HADOOP-12916: - GitHub user xiaoyuyao opened a pull request: https://github.com/apache/hadoop/pull/86 HADOOP-12916 Create a github PR for code review of v04 patch on the JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/xiaoyuyao/hadoop HADOOP-12916 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/86.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #86 commit 93cf7377980a69f8b5b22453419a95b58a19c2aa Author: Xiaoyu YaoDate: 2016-03-26T02:45:48Z HADOOP-12916.patch.04 > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207450#comment-15207450 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 43s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 13s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 55s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 36s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 11s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 50s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 50s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s {color} | {color:green} hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 52s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 41s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_74. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 59s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 21s {color} | {color:red} Patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 59m 38s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_74 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:fbe3e86 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12794848/HADOOP-12916.04.patch | | JIRA Issue | HADOOP-12916 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 5f5716499f03 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207367#comment-15207367 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 4s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 29s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 18s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 40s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 36s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 42s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 42s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s {color} | {color:green} hadoop-common-project/hadoop-common: patch generated 0 new + 523 unchanged - 45 fixed = 523 total (was 568) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 47s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 40s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_74. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 2s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 23s {color} | {color:red} Patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 61m 6s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_74 Failed junit tests | hadoop.net.TestClusterTopology | | JDK v1.8.0_74 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:fbe3e86 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12794822/HADOOP-12916.03.patch | | JIRA Issue | HADOOP-12916 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207177#comment-15207177 ] Tsz Wo Nicholas Sze commented on HADOOP-12916: -- The new patch looks good. One last comment: - Since averageResponseTimeInLastWindow and callCountInLastWindow are atomic arrays, we can remove synchronized (this) when accessing them. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch, HADOOP-12916.03.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202987#comment-15202987 ] Tsz Wo Nicholas Sze commented on HADOOP-12916: -- Thanks for the update. I think averageResponseTimeInLastWindow and callCountInLastWindow probably need to be declared with AtomicLongArray and AtomicDoubleArray since we also need to synchronize individual values in the arrays. Patch looks good other than that. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch, > HADOOP-12916.02.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202904#comment-15202904 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 20s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 15s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 13s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 5s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 43s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 49s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 9s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 9s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 8s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 8s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 26s {color} | {color:red} hadoop-common-project/hadoop-common: patch generated 2 new + 523 unchanged - 44 fixed = 525 total (was 567) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 4s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 59s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 7s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_74. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 36s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 24s {color} | {color:red} Patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 77m 25s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_74 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:fbe3e86 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12794368/HADOOP-12916.02.patch | | JIRA Issue | HADOOP-12916 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 07f2e1d70d14 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200412#comment-15200412 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 9s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 40s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 8s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 44s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 34s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 53s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 17s {color} | {color:red} root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 53s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 44s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 17m 1s {color} | {color:red} root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 44s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 26s {color} | {color:red} hadoop-common-project/hadoop-common: patch generated 29 new + 548 unchanged - 19 fixed = 577 total (was 567) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 52s {color} | {color:red} hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 50s {color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_74. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 13s {color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 22s {color} | {color:red} Patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 58m 58s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-common-project/hadoop-common | | | Unread
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202609#comment-15202609 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 8s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 34s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 59s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 42s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 34s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 54s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 18s {color} | {color:red} root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 54s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 43s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 17m 1s {color} | {color:red} root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 43s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 26s {color} | {color:red} hadoop-common-project/hadoop-common: patch generated 4 new + 544 unchanged - 23 fixed = 548 total (was 567) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 49s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 35s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_74. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 44s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 22s {color} | {color:red} Patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 57m 54s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_74 Failed junit tests | hadoop.ha.TestZKFailoverController | | JDK v1.8.0_74 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202579#comment-15202579 ] Hadoop QA commented on HADOOP-12916: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 29s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 48s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 50s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 35s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s {color} | {color:green} trunk passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 40s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 51s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 19s {color} | {color:red} root-jdk1.8.0_74 with JDK v1.8.0_74 generated 15 new + 738 unchanged - 0 fixed = 753 total (was 738) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 51s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 41s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 17m 0s {color} | {color:red} root-jdk1.7.0_95 with JDK v1.7.0_95 generated 14 new + 734 unchanged - 0 fixed = 748 total (was 734) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 41s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 27s {color} | {color:red} hadoop-common-project/hadoop-common: patch generated 4 new + 544 unchanged - 23 fixed = 548 total (was 567) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 47s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} the patch passed with JDK v1.8.0_74 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 26s {color} | {color:red} hadoop-common in the patch failed with JDK v1.8.0_74. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 46s {color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 22s {color} | {color:red} Patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 57m 43s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_74 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | | JDK v1.7.0_95 Timed out junit tests | org.apache.hadoop.util.TestNativeLibraryChecker | \\
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202508#comment-15202508 ] Xiaoyu Yao commented on HADOOP-12916: - Got it. Update the patch with the priorityLevel passed for the second constructor. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202352#comment-15202352 ] Tsz Wo Nicholas Sze commented on HADOOP-12916: -- Thanks [~xyao]. Some comments on the patch. - In CallQueueManager.createScheduler, -* createScheduler should be static. -* Should priorityLevels be passed in the second try-catch? - Remove public from getPriorityLevel. - In parseNumLevels, print ns in the warning. - backOffByResponseTimeEnabled should be declared with a lower case boolean. - The following are not used. Let's remove them. -* RpcScheduler.getPriorityLevels() -* getAverageResponseTime(int level) -* setScheduler -* MetricsProxy.numLevels - DefaultRpcScheduler needs license header. - averageResponseTimeInLastWindow and callCountInLastWindow need to be synchronized. - Let's remove the unused import from Schedulable. > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12916) Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
[ https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202471#comment-15202471 ] Tsz Wo Nicholas Sze commented on HADOOP-12916: -- I mean ctor.newInstance() below. {code} + Constructor ctor = theClass.getDeclaredConstructor(int.class); + return ctor.newInstance(); {code} > Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff > -- > > Key: HADOOP-12916 > URL: https://issues.apache.org/jira/browse/HADOOP-12916 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch > > > Currently back off policy from HADOOP-10597 is hard coded to base on whether > call queue is full. This ticket is open to allow flexible back off policies > such as moving average of response time in RPC calls of different priorities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)