Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint
On 09/10/2013 04:58 PM, Mike Spreitzer wrote: > First, I'm a newbie here, wondering: is this the right place for > comments/questions on blueprints? Supposing it is... Yes, this is the right place. > I am referring to > https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension > > In my own research group we have experience with a few systems that do > something like that, and more (as, indeed, that blueprint explicitly > states that it is only the start of a longer roadmap). I would like to > highlight a couple of differences that alarm me. One is the general > overlap between groups. I am not saying this is wrong, but as a matter > of natural conservatism we have shied away from unnecessary > complexities. The only overlap we have done so far is hierarchical > nesting. As the instance-group-api-extension explicitly contemplates > groups of groups as a later development, this would cover the overlap > that we have needed. On the other hand, we already have multiple > "policies" attached to a single group. We have policies for a variety > of concerns, so some can combine completely or somewhat independently. > We also have relationships (of various sorts) between groups (as well > as between individuals, and between individuals and groups). The > policies and relationships, in general, are not simply names but also > have parameters. I'll let those driving this feature comment on this. I just wanted to confirm that you're in the right place. :-) -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint
From: Mike Spreitzer mailto:mspre...@us.ibm.com>> Reply-To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Date: Tuesday, September 10, 2013 11:58 PM To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint First, I'm a newbie here, wondering: is this the right place for comments/questions on blueprints? Supposing it is... [Gary Kotton] Yeah, as Russel said this is the correct place I am referring to https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension In my own research group we have experience with a few systems that do something like that, and more (as, indeed, that blueprint explicitly states that it is only the start of a longer roadmap). I would like to highlight a couple of differences that alarm me. One is the general overlap between groups. I am not saying this is wrong, but as a matter of natural conservatism we have shied away from unnecessary complexities. The only overlap we have done so far is hierarchical nesting. As the instance-group-api-extension explicitly contemplates groups of groups as a later development, this would cover the overlap that we have needed. On the other hand, we already have multiple "policies" attached to a single group. We have policies for a variety of concerns, so some can combine completely or somewhat independently. We also have relationships (of various sorts) between groups (as well as between individuals, and between individuals and groups). The policies and relationships, in general, are not simply names but also have parameters. [Gary Kotton] The instance groups was meant to be the first step towards what we had presented in Portland. Please look at the presentation that we gave an this may highlight what the aims were: https://docs.google.com/presentation/d/1oDXEab2mjxtY-cvufQ8f4cOHM0vIp4iMyfvZPqg8Ivc/edit?usp=sharing. Sadly for this release we did not manage to get the instance groups through (it was an issue of timing and bad luck). We will hopefully get this though in the first stages of the I cycle and then carry on building on it as it has a huge amount of value for OpenStack. It will be great if you can also participate in the discussions. Thanks, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint
Yes, I've seen that material. In my group we have worked larger and more complex examples. I have a proposed breakout session at the Hong Kong summit to talk about one, you might want to vote for it. The URL is http://www.openstack.org/summit/openstack-summit-hong-kong-2013/become-a-speaker/TalkDetails/109 and the title is "Continuous Delivery of Lotus Connections on OpenStack". We used our own technology to do the scheduling (make placement decisions) and orchestration, calling Nova and Quantum to carry out the decisions our software made. Above the OpenStack infrastructure we used two layers of our own software, one focused on infrastructure and one adding concerns for the software running on that infrastructure. Each used its own language for a whole topology AKA pattern AKA application AKA cluster. For example, our pattern has 16 VMs running the WebSphere application server, organized into four homogenous groups (members are interchangeable) of four each. For each group, we asked that it both (a) be spread across at least two racks, with no more than half the VMs on any one rack and (b) have no two VMs on the same hypervisor. You can imagine how this would involve multiple levels of grouping and relationships between groups (and you will probably be surprised by the particulars). We also included information on licensed products, so that the placement decision can optimize license cost (for the IBM "sub-capacity" licenses, placement of VMs can make a cost difference). Thus, multiple policies per thing. We are now extending that example to include storage, and we are also working examples with Hadoop. Regards, Mike From: Gary Kotton To: OpenStack Development Mailing List , Date: 09/11/2013 06:06 AM Subject: Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint From: Mike Spreitzer Reply-To: OpenStack Development Mailing List < openstack-dev@lists.openstack.org> Date: Tuesday, September 10, 2013 11:58 PM To: OpenStack Development Mailing List Subject: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint First, I'm a newbie here, wondering: is this the right place for comments/questions on blueprints? Supposing it is... [Gary Kotton] Yeah, as Russel said this is the correct place I am referring to https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension In my own research group we have experience with a few systems that do something like that, and more (as, indeed, that blueprint explicitly states that it is only the start of a longer roadmap). I would like to highlight a couple of differences that alarm me. One is the general overlap between groups. I am not saying this is wrong, but as a matter of natural conservatism we have shied away from unnecessary complexities. The only overlap we have done so far is hierarchical nesting. As the instance-group-api-extension explicitly contemplates groups of groups as a later development, this would cover the overlap that we have needed. On the other hand, we already have multiple "policies" attached to a single group. We have policies for a variety of concerns, so some can combine completely or somewhat independently. We also have relationships (of various sorts) between groups (as well as between individuals, and between individuals and groups). The policies and relationships, in general, are not simply names but also have parameters. [Gary Kotton] The instance groups was meant to be the first step towards what we had presented in Portland. Please look at the presentation that we gave an this may highlight what the aims were: https://docs.google.com/presentation/d/1oDXEab2mjxtY-cvufQ8f4cOHM0vIp4iMyfvZPqg8Ivc/edit?usp=sharing . Sadly for this release we did not manage to get the instance groups through (it was an issue of timing and bad luck). We will hopefully get this though in the first stages of the I cycle and then carry on building on it as it has a huge amount of value for OpenStack. It will be great if you can also participate in the discussions. Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint
Mike, You mention "We are now extending that example to include storage, and we are also working examples with Hadoop. " In the context of your examples / scenarios, do these placement decisions consider storage performance and capacity on a physical node? For example: Based on application needs, and IOPS, latency requirements - carving out a SSD storage or a traditional spinning disk block volume? Or say for cost-efficiency reasons using SSD caching on Hadoop name nodes? I'm investigating a) Per node PCIe SSD deployment need in Openstack environment / Hadoop environment and ,b) selected node SSD caching, specifically for OpenStack Cinder. Hope this is the right forum to ask this question. rgds, S On Sep 12, 2013, at 12:29 AM, Mike Spreitzer wrote: > Yes, I've seen that material. In my group we have worked larger and more > complex examples. I have a proposed breakout session at the Hong Kong summit > to talk about one, you might want to vote for it. The URL is > http://www.openstack.org/summit/openstack-summit-hong-kong-2013/become-a-speaker/TalkDetails/109 > and the title is "Continuous Delivery of Lotus Connections on OpenStack". > We used our own technology to do the scheduling (make placement decisions) > and orchestration, calling Nova and Quantum to carry out the decisions our > software made. Above the OpenStack infrastructure we used two layers of our > own software, one focused on infrastructure and one adding concerns for the > software running on that infrastructure. Each used its own language for a > whole topology AKA pattern AKA application AKA cluster. For example, our > pattern has 16 VMs running the WebSphere application server, organized into > four homogenous groups (members are interchangeable) of four each. For each > group, we asked that it both (a) be spread across at least two racks, with no > more than half the VMs on any one rack and (b) have no two VMs on the same > hypervisor. You can imagine how this would involve multiple levels of > grouping and relationships between groups (and you will probably be surprised > by the particulars). We also included information on licensed products, so > that the placement decision can optimize license cost (for the IBM > "sub-capacity" licenses, placement of VMs can make a cost difference). Thus, > multiple policies per thing. We are now extending that example to include > storage, and we are also working examples with Hadoop. > > Regards, > Mike > > > > From: Gary Kotton > To:OpenStack Development Mailing List > , > Date: 09/11/2013 06:06 AM > Subject:Re: [openstack-dev] [heat] Comments/questions on the > instance-group-api-extension blueprint > > > > > > From: Mike Spreitzer > Reply-To: OpenStack Development Mailing List > > Date: Tuesday, September 10, 2013 11:58 PM > To: OpenStack Development Mailing List > Subject: [openstack-dev] [heat] Comments/questions on the > instance-group-api-extension blueprint > > First, I'm a newbie here, wondering: is this the right place for > comments/questions on blueprints? Supposing it is... > > [Gary Kotton] Yeah, as Russel said this is the correct place > > I am referring to > https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension > > In my own research group we have experience with a few systems that do > something like that, and more (as, indeed, that blueprint explicitly states > that it is only the start of a longer roadmap). I would like to highlight a > couple of differences that alarm me. One is the general overlap between > groups. I am not saying this is wrong, but as a matter of natural > conservatism we have shied away from unnecessary complexities. The only > overlap we have done so far is hierarchical nesting. As the > instance-group-api-extension explicitly contemplates groups of groups as a > later development, this would cover the overlap that we have needed. On the > other hand, we already have multiple "policies" attached to a single group. > We have policies for a variety of concerns, so some can combine completely or > somewhat independently. We also have relationships (of various sorts) > between groups (as well as between individuals, and between individuals and > groups). The policies and relationships, in general, are not simply names > but also have parameters. > > [Gary Kotton] The instance groups was meant to be the first step towards what > we had presented in Portland. Please look at the presentation that we gave an > this may highlight what the aims were: > https://docs.google.com/presentation/d/1oDXEab2mjxtY-cvufQ8f4cOHM0vIp4iMyfvZP
Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint
Hi, For some reason I am unable to access your proceed talk. I am not 100% sure but I think that the voting may be closed. We have weekly scheduling meetings (https://wiki.openstack.org/wiki/Meetings#Scheduler_Sub-group_meeting). It would be nice if you could attend and it will give you a platform to raise and share ideas with the rest of the guys in the community. At the moment the scheduling subgroup is working on our ideas for the design summit sessions. Please see https://etherpad.openstack.org/IceHouse-Nova-Scheduler-Sessions Thanks Gary From: Mike Spreitzer mailto:mspre...@us.ibm.com>> Reply-To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Date: Wednesday, September 11, 2013 9:59 PM To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint Yes, I've seen that material. In my group we have worked larger and more complex examples. I have a proposed breakout session at the Hong Kong summit to talk about one, you might want to vote for it. The URL is http://www.openstack.org/summit/openstack-summit-hong-kong-2013/become-a-speaker/TalkDetails/109 and the title is "Continuous Delivery of Lotus Connections on OpenStack". We used our own technology to do the scheduling (make placement decisions) and orchestration, calling Nova and Quantum to carry out the decisions our software made. Above the OpenStack infrastructure we used two layers of our own software, one focused on infrastructure and one adding concerns for the software running on that infrastructure. Each used its own language for a whole topology AKA pattern AKA application AKA cluster. For example, our pattern has 16 VMs running the WebSphere application server, organized into four homogenous groups (members are interchangeable) of four each. For each group, we asked that it both (a) be spread across at least two racks, with no more than half the VMs on any one rack and (b) have no two VMs on the same hypervisor. You can imagine how this would involve multiple levels of grouping and relationships between groups (and you will probably be surprised by the particulars). We also included information on licensed products, so that the placement decision can optimize license cost (for the IBM "sub-capacity" licenses, placement of VMs can make a cost difference). Thus, multiple policies per thing. We are now extending that example to include storage, and we are also working examples with Hadoop. Regards, Mike From:Gary Kotton mailto:gkot...@vmware.com>> To:OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>>, Date: 09/11/2013 06:06 AM Subject: Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint From: Mike Spreitzer mailto:mspre...@us.ibm.com>> Reply-To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Date: Tuesday, September 10, 2013 11:58 PM To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint First, I'm a newbie here, wondering: is this the right place for comments/questions on blueprints? Supposing it is... [Gary Kotton] Yeah, as Russel said this is the correct place I am referring to https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension In my own research group we have experience with a few systems that do something like that, and more (as, indeed, that blueprint explicitly states that it is only the start of a longer roadmap). I would like to highlight a couple of differences that alarm me. One is the general overlap between groups. I am not saying this is wrong, but as a matter of natural conservatism we have shied away from unnecessary complexities. The only overlap we have done so far is hierarchical nesting. As the instance-group-api-extension explicitly contemplates groups of groups as a later development, this would cover the overlap that we have needed. On the other hand, we already have multiple "policies" attached to a single group. We have policies for a variety of concerns, so some can combine completely or somewhat independently. We also have relationships (of various sorts) between groups (as well as between individuals, and between individuals and groups). The policies and relationships, in general, are not simply names but also have parameters. [Gary Kotton] The instance groups was meant to be the first step towards what we had presented in Portland. Please look at the presentation that we gave an this may highlight what the aims were: https://docs.google.com/presentation/d/1oDXEab2mjxtY-cvufQ8f4cOHM0
Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint
We are currently explicitly considering location and space. For example, a template can require that a volume be in a disk that is directly attached to the machine hosting the VM to which the volume is attached. Spinning rust bandwidth is much trickier because it is not something you can simply add up when you combine workloads. The IOPS, as well as the B/S, that a disk will deliver depends on the workload mix on that disk. While the disk may deliver X IOPS when serving only application A, and Y when serving only application B, you cannot conclude that it will serve (X+Y)/2 when serving (A+B)/2. While we hope to do better in the future, we currently handle disk bandwidth in non-quantitative ways. One is that a template may request that a volume be placed such that it does not compete with any other volume (i.e., is the only one on its disk). Another is that a template may specify a "type" for a volume, which effectively maps to a Cinder volume type that has been pre-defined to correspond to a QoS defined in an enterprise storage subsystem. The choice between fast&expensive vs slow&cheap storage is currently left to higher layers. That could be pushed down, supposing there is a suitably abstract yet accurate way of describing how the tradeoff choice should be made. I think Savanna people are on this list too, so I presume it's a good place for this discussion. Thanks, Mike From: shalz To: OpenStack Development Mailing List , Date: 09/11/2013 09:55 PM Subject: Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint Mike, You mention "We are now extending that example to include storage, and we are also working examples with Hadoop. " In the context of your examples / scenarios, do these placement decisions consider storage performance and capacity on a physical node? For example: Based on application needs, and IOPS, latency requirements - carving out a SSD storage or a traditional spinning disk block volume? Or say for cost-efficiency reasons using SSD caching on Hadoop name nodes? I'm investigating a) Per node PCIe SSD deployment need in Openstack environment / Hadoop environment and ,b) selected node SSD caching, specifically for OpenStack Cinder. Hope this is the right forum to ask this question. rgds, S ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Comments/questions on the instance-group-api-extension blueprint
Gary Kotton wrote on 09/12/2013 05:40:59 AM: > From: Gary Kotton > To: OpenStack Development Mailing List , > Date: 09/12/2013 05:46 AM > Subject: Re: [openstack-dev] [heat] Comments/questions on the > instance-group-api-extension blueprint > > Hi, > For some reason I am unable to access your proceed talk. I am not > 100% sure but I think that the voting may be closed. We have weekly > scheduling meetings (https://wiki.openstack.org/wiki/ > Meetings#Scheduler_Sub-group_meeting). It would be nice if you could > attend and it will give you a platform to raise and share ideas with > the rest of the guys in the community. > At the moment the scheduling subgroup is working on our ideas for > the design summit sessions. Please see https:// > etherpad.openstack.org/IceHouse-Nova-Scheduler-Sessions > Thanks > Gary Worse yet, I know of no way to navigate to a list of design summit proposals. What am I missing? The scheduler group meeting conflicts with another meeting that I already have and will be difficult to move. I will see what I can do asynchronously. Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
