[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599631#comment-16599631 ] Daisuke Kobayashi edited comment on HBASE-6721 at 9/1/18 1:28 PM: -- While studying the code, I happen to find HBASE-19799 where it added UI for RSGroups. Hence I may close this request then. Thank you for having a look, sir [~stack]! was (Author: daisuke.kobayashi): While studying the code, I happen to find HBASE-19799 where it added UI for RSGroups. Hence I may close this request then. Thank you for having a look, sir! > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Labels: hbase-6721 > Fix For: 2.0.0 > > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > hbase-6721-v27.patch, hbase-6721-v27.patch, hbase-6721-v27.patch.txt, > hbase-6721-v28.patch, hbase-6721-v28.patch, hbase-6721-v29.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595769#comment-16595769 ] Daisuke Kobayashi edited comment on HBASE-6721 at 8/29/18 12:09 AM: bq. Open subtask or new issue? Ok, I will open a new subtask of this. bq. Have you tried it sir? Due to a build issue on my local instance, I have not yet completed it. Thanks [~stack]! was (Author: daisuke.kobayashi): bq. Open subtask or new issue? Ok, I will open a new subtask of this. bq. Have you tried it sir? Due to a build issue on my local instance, I have yet completed it. Thanks [~stack]! > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Labels: hbase-6721 > Fix For: 2.0.0 > > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > hbase-6721-v27.patch, hbase-6721-v27.patch, hbase-6721-v27.patch.txt, > hbase-6721-v28.patch, hbase-6721-v28.patch, hbase-6721-v29.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15069872#comment-15069872 ] Elliott Clark edited comment on HBASE-6721 at 12/23/15 5:03 PM: >From what I can see most of my comments still stand. It's still built into the >default client. It's still in the main module. Though I do appreciate that >it's a co-processor. If we can move the co-processor to a different module and move the methods off the main admin classes I would be fine with it. was (Author: eclark): >From what I can see most of my comments still stand. It's still built into the >default client. It's still in the main module. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, immediateAssignments Sequence Diagram.svg, > randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg, > roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876096#comment-14876096 ] Andrew Purtell edited comment on HBASE-6721 at 9/18/15 6:31 PM: bq. Andrew Purtell The new patch is based off of current master. Should we just replace the current branch with a new branch to go with the new patch? I would say so if you want it. bq. Also was wondering since this is now cp-based. Do we still need a branch? Up to you. We can rebase or delete, either way let me know. Some issues with the current security integration. Coprocessors can't call into the internals of other coprocessors. I understand why this was done, but we can't have it. Coprocessors calling into the internals of other coprocessors, this is a non-negotiable point for the sake of sanity in maintenance of separate optional extensions. It's a catch-22 imposed on this change by the requirement it be a coprocessor only implementation. What I would suggest is introduce into the MasterObserver API hooks for the group admin APIs. Let the implementation of the group admin APIs and the authoritative security decisions both be separate mix-ins provided by different coprocessors. There needs to be common plumbing for the two. That belongs in MasterObserver. The plumbing could look like: - MasterObserver support for pre/post group admin API action hooks - In GroupAdminEndpoint, get the coprocessor host with getMasterCoprocessorHost() - Invoke the public (technically, LimitedPrivate(COPROC)) APIs for pre/post group admin API actions. - AccessController implements the new MasterObserver APIs to provide security for the group admin APIs. This is much more in spirit with current interfaces and audience scoping. It decouples GroupAdminEndpoint from AccessController. (If the AC is not installed, no harm, no NPEs, no security checking (by intention), it's all good.) It also addresses concerns about zero impact in the default case. Those upcalls will never be made unless the GroupAdminEndpoint is installed. was (Author: apurtell): bq. Andrew Purtell The new patch is based off of current master. Should we just replace the current branch with a new branch to go with the new patch? I would say so if you want it. bq. Also was wondering since this is now cp-based. Do we still need a branch? Up to you. We can rebase or delete, either way let me know. Some issues with the current security integration. Coprocessors can't call into the internals of other coprocessors. I understand why this was done, but we can't have it. Coprocessors calling into the internals of other coprocessors, this is a non-negotiable point for the sake of sanity in maintenance of separate optional extensions. It's a catch-22 imposed on this change by the requirement it be a coprocessor only implementation. What I would suggest is introduce into the MasterObserver API hooks for the group admin APIs. Let the implementation of the group admin APIs and the authoritative security decisions both be separate mix-ins provided by different coprocessors. There needs to be common plumbing for the two. That belongs in MasterObserver. The plumbing could look like: - MasterObserver support for pre/post group admin API action hooks - In GroupAdminEndpoint, get the coprocessor host with getMasterCoprocessorHost() - Invoke the public (technically, LimitedPrivate(COPROC)) APIs for pre/post group admin API actions. This is much more in spirit with current interfaces and audience scoping. It also addresses concerns about zero impact in the default case. Those upcalls will never be made unless the GroupAdminEndpoint is installed. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_8.patch, HBASE-6721_9.patch, HBASE-6721_9.patch, > HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721_94_2.patch, > HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94_4.patch, > HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, > HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Seque
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650440#comment-14650440 ] Andrew Purtell edited comment on HBASE-6721 at 8/1/15 4:37 PM: --- Thanks [~toffer]. I applied your latest patch to master and pushed the result as new branch "hbase-6721". I checked that it compiled before pushing but didn't run tests. Do you need this for 0.98? If so, we can do another branch "hbase-6721-0.98" with a backport and rebase it at every RC. Or branch-1. was (Author: apurtell): Thanks [~toffer]. I applied your latest patch to master and pushed the result as new branch "hbase-6721". I checked that it compiled before pushing but didn't run tests. Do you need this for 0.98? If so, we can do another branch "hbase-6721-0.98" with a backport and rebase it at every RC. Or one of the 1.x-es > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Attachments: 6721-master-webUI.patch, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_10.patch, HBASE-6721_11.patch, HBASE-6721_8.patch, > HBASE-6721_9.patch, HBASE-6721_9.patch, HBASE-6721_94.patch, > HBASE-6721_94.patch, HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_3.patch, HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, > HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650440#comment-14650440 ] Andrew Purtell edited comment on HBASE-6721 at 8/1/15 4:37 PM: --- Thanks [~toffer]. I applied your latest patch to master and pushed the result as new branch "hbase-6721". I checked that it compiled before pushing but didn't run tests. Do you need this for 0.98? If so, we can do another branch "hbase-6721-0.98" with a backport and rebase it at every RC. Or one of the 1.x-es was (Author: apurtell): Thanks [~toffer]. I applied your latest patch to master and pushed the result as new branch "hbase-6721". I checked that it compiled before pushing but didn't run tests. Do you need this for 0.98? If so, we can do another branch "hbase-6721-0.98" with a backport and rebase it at every RC. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Attachments: 6721-master-webUI.patch, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_10.patch, HBASE-6721_11.patch, HBASE-6721_8.patch, > HBASE-6721_9.patch, HBASE-6721_9.patch, HBASE-6721_94.patch, > HBASE-6721_94.patch, HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_3.patch, HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, > HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365417#comment-14365417 ] Andrew Purtell edited comment on HBASE-6721 at 3/17/15 5:08 PM: It could be useful to have a list of pros and cons for coprocessor implementation vs. core integration. Are we considering core integration for mainly code aesthetic reasons, for example, or are there significant obstacles imposed by a coprocessor implementations strategy? was (Author: apurtell): It could be useful to have a list of pros and cons for coprocessor implementation vs. core integration. Are we considering core integration as mainly code aesthetic reasons mostly, for example, or are there significant obstacles imposed by a coprocessor implementations strategy? > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Vandana Ayyalasomayajula > Attachments: 6721-master-webUI.patch, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_10.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, HBASE-6721_trunk2.patch > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13744080#comment-13744080 ] Andrew Purtell edited comment on HBASE-6721 at 8/19/13 6:15 PM: Thanks for clearing it up guys. Edit: Would be good if we can get this into 0.98 and also a 0.96 minor, then there's a great rolling upgrade story. Please let me know if I can be helpful there. was (Author: apurtell): Thanks for clearing it up guys. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Vandana Ayyalasomayajula > Fix For: 0.96.0 > > Attachments: 6721-master-webUI.patch, HBASE-6721_10.patch, > HBASE-6721_8.patch, HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_3.patch, HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, > HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, HBASE-6721_94.patch, > HBASE-6721_94.patch, HBASE-6721_9.patch, HBASE-6721_9.patch, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492474#comment-13492474 ] Andrew Purtell edited comment on HBASE-6721 at 11/7/12 4:27 PM: bq. The implementation is a bit clunky but probably the prolly the best choice if we want to keep the data in the tables. [~toffer] Keeping the group assignment data in a table (at least the authoritative record of it) seems appropriate and consistent with feedback. Have you also considered adding a state mirror in ZK to avoid the need for random assignment of catalog tables and the group table if it is available on (re)start? FYI, looks like the _3 patch picks up unrelated work. Edit: See my above recent comment in this regard. was (Author: apurtell): bq. The implementation is a bit clunky but probably the prolly the best choice if we want to keep the data in the tables. [~toffer] Keeping the group assignment data in a table (at least the authoritative record of it) seems appropriate and consistent with feedback. Have you also considered adding a state mirror in ZK to avoid the need for random assignment of catalog tables and the group table if it is available on (re)start? FYI, looks like the _3 patch picks up unrelated work. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Vandana Ayyalasomayajula > Fix For: 0.96.0 > > Attachments: HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, > HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721-DesigDoc.pdf > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490754#comment-13490754
]
Andrew Purtell edited comment on HBASE-6721 at 11/5/12 5:27 PM:
Regarding storing the group assignment information in ZooKeeper, I think this
is a good strategy. You can optimize for the case where HBase (re)starts with
valid group assignment data available in ZK. Therefore you can avoid some
bootstrapping challenges. However, you must persist the group assignment
information into a table, like how security does with {{_acl_}}. After starting
up, if in the uncommon case there is group assignment information in the
{{_group_}} (or similar) table that is not in sync with that in ZK (perhaps
because ZK state was cleared), you should update ZK data accordingly. From
there, there are a couple of options:
- WARN the administrator that the table assignments should be updated via
disable and enable.
- Automatically trigger reassignment via disable and enable.
- Region moves (if the assignment information is available - trunk only)
Edit: Fix incorrect use of markup.
was (Author: apurtell):
Regarding storing the group assignment information in ZooKeeper, I think
this is a good strategy. You can optimize for the case where HBase (re)starts
with valid group assignment data available in ZK. Therefore you can avoid some
bootstrapping challenges. However, you must persist the group assignment
information into a table, like how security does with "__acl__". After starting
up, if in the uncommon case there is group assignment information in the
"__group__" (or similar) table that is not in sync with that in ZK (perhaps
because ZK state was cleared), you should update ZK data accordingly. From
there, there are a couple of options:
- WARN the administrator that the table assignments should be updated via
disable and enable.
- Automatically trigger reassignment via disable and enable.
- Region moves (if the assignment information is available - trunk only)
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Vandana Ayyalasomayajula
> Fix For: 0.96.0
>
> Attachments: HBASE-6721_94_2.patch, HBASE-6721_94.patch,
> HBASE-6721_94.patch, HBASE-6721-DesigDoc.pdf
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488082#comment-13488082 ] Andrew Purtell edited comment on HBASE-6721 at 10/31/12 6:51 PM: - The current attached patch for 0.94 makes a ton of changes in o.a.h.h.master. This stuff should all be pulled out into a separate package that can be put into a Maven module like done with security in 0.94. What changes to core code remain after that? was (Author: apurtell): The current attached patch makes a ton of changes in o.a.h.h.master. This stuff should all be pulled out into a separate package that can be put into a Maven module like done with security in 0.94. What changes to core code remain after that? > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Vandana Ayyalasomayajula > Fix For: 0.96.0 > > Attachments: HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721-DesigDoc.pdf > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
