[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated HBASE-21439: -- Labels: balancer (was: ) > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Labels: balancer > Fix For: 3.0.0, 1.3.3, 2.2.0, 2.0.3, 1.4.9, 2.1.2 > > Attachments: HBASE-21439-addendum-master.patch, > HBASE-21439-branch-1.patch, HBASE-21439-branch-1.patch, > HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21439: --- Fix Version/s: (was: 1.5.0) > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Fix For: 3.0.0, 1.3.3, 2.2.0, 2.0.3, 1.4.9, 2.1.2 > > Attachments: HBASE-21439-addendum-master.patch, > HBASE-21439-branch-1.patch, HBASE-21439-branch-1.patch, > HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21439: --- Resolution: Fixed Fix Version/s: 2.1.2 1.4.9 2.0.3 2.2.0 1.3.3 1.5.0 Status: Resolved (was: Patch Available) > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.0.3, 1.4.9, 2.1.2 > > Attachments: HBASE-21439-addendum-master.patch, > HBASE-21439-branch-1.patch, HBASE-21439-branch-1.patch, > HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21439: --- Attachment: HBASE-21439-branch-1.patch > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21439-addendum-master.patch, > HBASE-21439-branch-1.patch, HBASE-21439-branch-1.patch, > HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-21439: Attachment: HBASE-21439-addendum-master.patch Status: Patch Available (was: Reopened) That's weird the build didn't pick it up, I guess by default it doesn't run RSGroup module tests? This patch should fix the test, it's a similar fix as for the other unit test. > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 2.0.2, 1.3.2.1 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21439-addendum-master.patch, > HBASE-21439-branch-1.patch, HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21439: --- Attachment: HBASE-21439-branch-1.patch > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21439-branch-1.patch, HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21439: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Thanks for the patch, Ben > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-21439: Attachment: HBASE-21439-master.patch Status: Patch Available (was: Open) New patch that fixes the style issues. > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 2.0.2, 1.3.2.1 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Attachments: HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-21439: Status: Open (was: Patch Available) > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 2.0.2, 1.3.2.1 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-21439: Attachment: (was: HBASE-21439-master.patch) > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-21439: Attachment: (was: HBASE-21439-master.patch) > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Attachments: HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-21439: Attachment: HBASE-21439-master.patch Status: Patch Available (was: Open) > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 2.0.2, 1.3.2.1 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Attachments: HBASE-21439-master.patch, HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-21439: Attachment: HBASE-21439-master.patch > StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost > functions > - > > Key: HBASE-21439 > URL: https://issues.apache.org/jira/browse/HBASE-21439 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 1.3.2.1, 2.0.2 >Reporter: Ben Lau >Assignee: Ben Lau >Priority: Major > Attachments: HBASE-21439-master.patch > > > In StochasticLoadBalancer.updateRegionLoad() the region loads are being put > into the map with Bytes.toString(regionName). > First, this is a problem because Bytes.toString() assumes that the byte array > is a UTF8 encoded String but there is no guarantee that regionName bytes are > legal UTF8. > Secondly, in BaseLoadBalancer.registerRegion, we are reading the region loads > out of the load map not using Bytes.toString() but using > region.getRegionNameAsString() and region.getEncodedName(). So the load > balancer will not see or use any of the cluster's RegionLoad history. > There are 2 primary ways to solve this issue, assuming we want to stay with > String keys for the load map (seems reasonable to aid debugging). We can > either fix updateRegionLoad to store the regionName as a string properly or > we can update both the reader & writer to use a new common valid String > representation. > Will post a patch assuming we want to pursue the original intention, i.e. > store regionNameAsAString for the loadmap key, but I'm open to fixing this a > different way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
