[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227787#comment-16227787 ] Chia-Ping Tsai commented on HBASE-17706: I had already fixed it in HBASE-18480. Will close this jira as duplicate. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer > Labels: patch > Fix For: 2.0.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227783#comment-16227783 ] stack commented on HBASE-17706: --- Should we up the priority on this [~chia7712]? Pull it into beta-1? > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer > Labels: patch > Fix For: 2.0.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106222#comment-16106222 ] Chia-Ping Tsai commented on HBASE-17706: The wrong value of numMaxRegionsPerTable almost destroys the action of MOVE_REGION since the new cost is always higher than origin cost. As for me, the priority of this bug should be major or higher... Ping [~kahliloppenheimer] If you are busy, I can give a patch. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Fix For: 2.0.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069149#comment-16069149
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 1m 47s{color}
| {color:red} HBASE-17706 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-17706 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12859119/HBASE-17706-07.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7403/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>Reporter: Kahlil Oppenheimer
>Assignee: Kahlil Oppenheimer
>Priority: Minor
> Labels: patch
> Fix For: 2.0.0
>
> Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch,
> HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch,
> HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch
>
>
> We noticed while running unit tests that the TableSkewCostFunction computed
> cost did not change as the balancer ran and simulated moves across the
> cluster. After investigating, we found that this happened in particular when
> the cluster started out with at least one table very strongly skewed.
> We noticed that the TableSkewCostFunction depends on a field of the
> BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field
> is not properly maintained as regionMoves are simulated for the cluster. The
> field only ever increases as the maximum number of regions per table
> increases, but it does not decrease as the maximum number per table goes down.
> This patch corrects that behavior so that the field is accurately maintained,
> and thus the TableSkewCostFunction produces a more correct value as the
> balancer runs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069138#comment-16069138 ] stack commented on HBASE-17706: --- What you want to do w/ this [~kahliloppenheimer] sir? Thanks. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Fix For: 2.0.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929552#comment-15929552 ] Hudson commented on HBASE-17706: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2687 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2687/]) HBASE-17706 TableSkewCostFunction improperly computes max skew - revert (tedyu: rev a69c23abfeed9246f308bf470b72a9a8afa46f5d) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Fix For: 2.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929329#comment-15929329 ] Ted Yu commented on HBASE-17706: Reverted from master since this was on top of HBASE-17707 Can you submit patch (not based on HBASE-17707) for another QA run ? We can get this in if tests pass. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Fix For: 2.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929184#comment-15929184 ] Hudson commented on HBASE-17706: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2685 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2685/]) HBASE-17706 TableSkewCostFunction improperly computes max skew (tedyu: rev edbd0e494d7abaf50319b7650e350d52b195fcc9) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Fix For: 2.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928647#comment-15928647 ] Ted Yu commented on HBASE-17706: Integrated to master. Will resolve after trunk build finishes. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928575#comment-15928575
]
Hadoop QA commented on HBASE-17706:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 12s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 110m 26s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 151m 13s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12859119/HBASE-17706-07.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 4aa4e0485903 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e67eb6c |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6118/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6118/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928165#comment-15928165
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s {color}
| {color:red} HBASE-17706 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12859108/HBASE-17706-06.patch |
| JIRA Issue | HBASE-17706 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6117/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>Reporter: Kahlil Oppenheimer
>Assignee: Kahlil Oppenheimer
>Priority: Minor
> Labels: patch
> Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch,
> HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch,
> HBASE-17706-06.patch, HBASE-17706.patch
>
>
> We noticed while running unit tests that the TableSkewCostFunction computed
> cost did not change as the balancer ran and simulated moves across the
> cluster. After investigating, we found that this happened in particular when
> the cluster started out with at least one table very strongly skewed.
> We noticed that the TableSkewCostFunction depends on a field of the
> BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field
> is not properly maintained as regionMoves are simulated for the cluster. The
> field only ever increases as the maximum number of regions per table
> increases, but it does not decrease as the maximum number per table goes down.
> This patch corrects that behavior so that the field is accurately maintained,
> and thus the TableSkewCostFunction produces a more correct value as the
> balancer runs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928158#comment-15928158 ] Kahlil Oppenheimer commented on HBASE-17706: [~tedyu] I just rebased the patch and resubmitted it. It might break on one test in TestStochasticLoadBalancer2 until HBASE-17707 is merged in (which contained a fix that affects this test). > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706-06.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927989#comment-15927989 ] Ted Yu commented on HBASE-17706: Mind rebasing the patch and submitting for QA run ? > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895361#comment-15895361
]
Hadoop QA commented on HBASE-17706:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 45s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 106m 15s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 154m 58s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855986/HBASE-17706-05.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5a2f8033f5bf 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 404a288 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5939/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5939/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895308#comment-15895308
]
Hadoop QA commented on HBASE-17706:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 53s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 102m 56s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 5s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855979/HBASE-17706-04.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 248115c301c8 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 404a288 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5938/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5938/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895168#comment-15895168 ] Kahlil Oppenheimer commented on HBASE-17706: Just fixed line lengths! > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, > HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895099#comment-15895099
]
Ted Yu commented on HBASE-17706:
Almost there.
{code}
+ // if old server had max num regions, assume (for now) max num regions
went down since we moved the region
{code}
Wrap long lines (there is one in test).
Limit to 100 characters per line.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>Reporter: Kahlil Oppenheimer
>Assignee: Kahlil Oppenheimer
>Priority: Minor
> Labels: patch
> Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch,
> HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706.patch
>
>
> We noticed while running unit tests that the TableSkewCostFunction computed
> cost did not change as the balancer ran and simulated moves across the
> cluster. After investigating, we found that this happened in particular when
> the cluster started out with at least one table very strongly skewed.
> We noticed that the TableSkewCostFunction depends on a field of the
> BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field
> is not properly maintained as regionMoves are simulated for the cluster. The
> field only ever increases as the maximum number of regions per table
> increases, but it does not decrease as the maximum number per table goes down.
> This patch corrects that behavior so that the field is accurately maintained,
> and thus the TableSkewCostFunction produces a more correct value as the
> balancer runs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895015#comment-15895015
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 20s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 107m 33s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 31s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855936/HBASE-17706-03.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e307a5cbe1e0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 678ad0e |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/5934/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5934/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5934/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
>
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894799#comment-15894799 ] Kahlil Oppenheimer commented on HBASE-17706: Just rebased the patch to be based off of the latest changes. Tests should pass now. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Assignee: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706-03.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893659#comment-15893659 ] Ted Yu commented on HBASE-17706: TestStochasticLoadBalancer2 still failed. Can you rebase the patch (now that we have new TableSkewCostFunction) ? > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, > HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891014#comment-15891014
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 31s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 29s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 128m 16s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855434/HBASE-17706-02.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 448f88513123 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 613bcb3 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/5904/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5904/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5904/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5904/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5904/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889050#comment-15889050
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 7s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 47s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 140m 57s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855210/HBASE-17706-01.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 8da468ef0e40 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 4a5eba5 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5880/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5880/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5880/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5880/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key:
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1523#comment-1523 ] Kahlil Oppenheimer commented on HBASE-17706: Sure! Just added a new test case and fixed one that was breaking. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-01.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888539#comment-15888539 ] Ted Yu commented on HBASE-17706: Is it possible to add (modify existing) test to show that fix works ? > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887513#comment-15887513
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 106m 36s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 155m 33s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855060/HBASE-17706.patch |
| JIRA Issue | HBASE-17706 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 52567df1329a 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f4e0ea2 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/5869/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/5869/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5869/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5869/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887313#comment-15887313 ] Kahlil Oppenheimer commented on HBASE-17706: OK, this one is based off master so hopefully it will work now. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887299#comment-15887299 ] Ted Yu commented on HBASE-17706: Please generate patch for master branch first. Once it passes review, you can attach backported patches. > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-branch-1.2.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887297#comment-15887297
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 7m 1s {color}
| {color:red} Docker failed to build yetus/hbase:d6626eb. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855054/HBASE-17706-branch-1.2.patch
|
| JIRA Issue | HBASE-17706 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5868/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>Reporter: Kahlil Oppenheimer
>Priority: Minor
> Labels: patch
> Attachments: HBASE-17706-branch-1.2.patch
>
>
> We noticed while running unit tests that the TableSkewCostFunction computed
> cost did not change as the balancer ran and simulated moves across the
> cluster. After investigating, we found that this happened in particular when
> the cluster started out with at least one table very strongly skewed.
> We noticed that the TableSkewCostFunction depends on a field of the
> BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field
> is not properly maintained as regionMoves are simulated for the cluster. The
> field only ever increases as the maximum number of regions per table
> increases, but it does not decrease as the maximum number per table goes down.
> This patch corrects that behavior so that the field is accurately maintained,
> and thus the TableSkewCostFunction produces a more correct value as the
> balancer runs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887290#comment-15887290 ] Kahlil Oppenheimer commented on HBASE-17706: The patch is based on branch-1.2. Is it OK to submit as such? > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706-branch-1.2.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887261#comment-15887261 ] Ted Yu commented on HBASE-17706: Please rebase patch on master branch. Is it possible to add (modify existing) test to show that fix works ? > TableSkewCostFunction improperly computes max skew > -- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 > kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. >Reporter: Kahlil Oppenheimer >Priority: Minor > Labels: patch > Attachments: HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed > cost did not change as the balancer ran and simulated moves across the > cluster. After investigating, we found that this happened in particular when > the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the > BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field > is not properly maintained as regionMoves are simulated for the cluster. The > field only ever increases as the maximum number of regions per table > increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, > and thus the TableSkewCostFunction produces a more correct value as the > balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew
[
https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887247#comment-15887247
]
Hadoop QA commented on HBASE-17706:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-17706 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12855051/HBASE-17706.patch |
| JIRA Issue | HBASE-17706 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5866/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> TableSkewCostFunction improperly computes max skew
> --
>
> Key: HBASE-17706
> URL: https://issues.apache.org/jira/browse/HBASE-17706
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>Reporter: Kahlil Oppenheimer
>Priority: Minor
> Labels: patch
> Attachments: HBASE-17706.patch
>
>
> We noticed while running unit tests that the TableSkewCostFunction computed
> cost did not change as the balancer ran and simulated moves across the
> cluster. After investigating, we found that this happened in particular when
> the cluster started out with at least one table very strongly skewed.
> We noticed that the TableSkewCostFunction depends on a field of the
> BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field
> is not properly maintained as regionMoves are simulated for the cluster. The
> field only ever increases as the maximum number of regions per table
> increases, but it does not decrease as the maximum number per table goes down.
> This patch corrects that behavior so that the field is accurately maintained,
> and thus the TableSkewCostFunction produces a more correct value as the
> balancer runs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
