The nova guys propose move Hyper to Magnum but not Nova as Hyper cannot fit into nova virt driver well.
As Hyper is now integrating with Kubernetes, I think that the integration point may be creating a kubernetes hyper bay with ironic driver. Thanks 2015-07-20 10:00 GMT+08:00 Kai Qiang Wu <wk...@cn.ibm.com>: > Hi Peng, > > As @Adrian pointed it out: > > *My fist suggestion is to find a way to make a nova virt driver for Hyper, > which could allow it to be used with all of our current Bay types in > Magnum.* > > > I remembered you or other guys in your company proposed one bp about nova > virt driver for Hyper. What's the status of the bp now? > Is it accepted by nova projects or cancelled ? > > > Thanks > > Best Wishes, > > -------------------------------------------------------------------------------- > Kai Qiang Wu (吴开强 Kennan) > IBM China System and Technology Lab, Beijing > > E-mail: wk...@cn.ibm.com > Tel: 86-10-82451647 > Address: Building 28(Ring Building), ZhongGuanCun Software Park, > No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China > 100193 > > -------------------------------------------------------------------------------- > Follow your heart. You are miracle! > > [image: Inactive hide details for Adrian Otto ---07/19/2015 11:18:02 > PM---Peng, You are not the first to think this way, and it's one o]Adrian > Otto ---07/19/2015 11:18:02 PM---Peng, You are not the first to think this > way, and it's one of the reasons we did not integrate Cont > > From: Adrian Otto <adrian.o...@rackspace.com> > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > Date: 07/19/2015 11:18 PM > Subject: Re: [openstack-dev] [magnum][bp] Power Magnum to run on metal > withHyper > ------------------------------ > > > > Peng, > > You are not the first to think this way, and it's one of the reasons we > did not integrate Containers with OpenStack in a meaningful way a full year > earlier. Please pay attention closely. > > 1) OpenStack's key influences care about two personas: 1.1) Cloud > Operators 1.2) Cloud Consumers. If you only think in terms of 1.2, then > your idea will get killed. Operators matter. > > 2) Cloud Operators need a consistent way to bill for the IaaS services the > provide. Nova emits all of the RPC messages needed to do this. Having a > second nova that does this slightly differently is a really annoying > problem that will make Operators hate the software. It's better to use > nova, have things work consistently, and plug in virt drivers to it. > > 3) Creation of a host is only part of the problem. That's the easy part. > Nova also does a bunch of other things too. For example, say you want to > live migrate a guest from one host to another. There is already > functionality in Nova for doing that. > > 4) Resources need to be capacity managed. We call this scheduling. Nova > has a pluggable scheduler to help with the placement of guests on hosts. > Magnum will not. > > 5) Hosts in a cloud need to integrate with a number of other services, > such as an image service, messaging, networking, storage, etc. If you only > think in terms of host creation, and do something without nova, then you > need to re-integrate with all of these things. > > Now, I probably left out examples of lots of other things that Nova does. > What I have mentioned us enough to make my point that there are a lot of > things that Magnum is intentionally NOT doing that we expect to get from > Nova, and I will block all code that gratuitously duplicates functionality > that I believe belongs in Nova. I promised our community I would not > duplicate existing functionality without a very good reason, and I will > keep that promise. > > Let's find a good way to fit Hyper with OpenStack in a way that best > leverages what exists today, and is least likely to be rejected. Please > note that the proposal needs to be changed from where it is today to > achieve this fit. > > My fist suggestion is to find a way to make a nova virt driver for Hyper, > which could allow it to be used with all of our current Bay types in Magnum. > > Thanks, > > Adrian > > > -------- Original message -------- > From: Peng Zhao <p...@hyper.sh> > Date: 07/19/2015 5:36 AM (GMT-08:00) > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > Subject: Re: [openstack-dev] [magnum][bp] Power Magnum to run on metal > withHyper > > Thanks Jay. > > Hongbin, yes, it will be a scheduling system, either swarm, k8s or mesos. > I just think bay isn't a must in this case, and we don't need nova to > provision BM hosts, which makes things more complicated imo. > > Peng > > > ------------------ Original ------------------ > *From: * "Jay Lau"<jay.lau....@gmail.com>; > *Date: * Sun, Jul 19, 2015 10:36 AM > *To: * "OpenStack Development Mailing List (not for usage questions)"< > openstack-dev@lists.openstack.org>; > *Subject: * Re: [openstack-dev] [magnum][bp] Power Magnum to run on metal > withHyper > > Hong Bin, > > I have some online discussion with Peng, seems hyper is now integrating > with Kubernetes and also have plan integrate with mesos for scheduling. > Once mesos integration finished, we can treat mesos+hyper as another kind > of bay. > > Thanks > > 2015-07-19 4:15 GMT+08:00 Hongbin Lu <*hongbin...@huawei.com* > <hongbin...@huawei.com>>: > > Peng, > > > > Several questions Here. You mentioned that HyperStack is a single big > “bay”. Then, who is doing the multi-host scheduling, Hyper or something > else? Were you suggesting to integrate Hyper with Magnum directly? Or you > were suggesting to integrate Hyper with Magnum indirectly (i.e. through > k8s, mesos and/or Nova)? > > > > Best regards, > > Hongbin > > > > *From:* Peng Zhao [mailto:*p...@hyper.sh* <p...@hyper.sh>] > * Sent:* July-17-15 12:34 PM > * To:* OpenStack Development Mailing List (not for usage questions) > > *Subject:* Re: [openstack-dev] [magnum][bp] Power Magnum to run on > metal with Hyper > > > > Hi, Adrian, Jay and all, > > > > There could be a much longer version of this, but let me try to > explain in a minimalist way. > > > > Bay currently has two modes: VM-based, BM-based. In both cases, Bay > helps to isolate different tenants' containers. In other words, bay is > single-tenancy. For BM-based bay, the single tenancy is a worthy tradeoff, > given the performance merits of LXC vs VM. However, for a VM-based bay, > there is no performance gain, but single tenancy seems a must, due to the > lack of isolation in container. Hyper, as a hypervisor-based substitute for > container, brings the much-needed isolation, and therefore enables > multi-tenancy. In HyperStack, we don't really need Ironic to provision > multiple Hyper bays. On the other hand, the entire HyperStack cluster is a > single big "bay". Pretty similar to how Nova works. > > > > Also, HyperStack is able to leverage Cinder, Neutron for SDS/SDN > functionality. So when someone submits a Docker Compose app, HyperStack > would launch HyperVMs and call Cinder/Neutron to setup the volumes and > network. The architecture is quite simple. > > > > Here are a blog I'd like to recommend: > > *https://hyper.sh/blog/post/2015/06/29/docker-hyper-and-the-end-of-guest-os.html* > > <https://hyper.sh/blog/post/2015/06/29/docker-hyper-and-the-end-of-guest-os.html> > > > > Let me know your questions. > > > > Thanks, > > Peng > > > > ------------------ Original ------------------ > > *From: * "Adrian Otto"<*adrian.o...@rackspace.com* > <adrian.o...@rackspace.com>>; > > *Date: * Thu, Jul 16, 2015 11:02 PM > > *To: * "OpenStack Development Mailing List (not for usage questions)"< > *openstack-dev@lists.openstack.org* <openstack-dev@lists.openstack.org>>; > > > *Subject: * Re: [openstack-dev] [magnum][bp] Power Magnum to run > onmetalwith Hyper > > > > Jay, > > > > Hyper is a substitute for a Docker host, so I expect it could work > equally well for all of the current bay types. Hyper’s idea of a “pod” and > a Kubernetes “pod” are similar, but different. I’m not yet convinced that > integrating Hyper host creation direct with Magnum (and completely > bypassing nova) is a good idea. It probably makes more sense to implement > use nova with the ironic dirt driver to provision Hyper hosts so we can use > those as substitutes for Bay nodes in our various Bay types. This would fit > in the place were we use Fedora Atomic today. We could still rely on nova > to do all of the machine instance management and accounting like we do > today, but produce bays that use Hyper instead of a Docker host. Everywhere > we currently offer CoreOS as an option we could also offer Hyper as an > alternative, with some caveats. > > > > There may be some caveats/drawbacks to consider before committing to a > Hyper integration. I’ll be asking those of Peng also on this thread, so > keep an eye out. > > > > Thanks, > > > > Adrian > > > On Jul 16, 2015, at 3:23 AM, Jay Lau <*jay.lau....@gmail.com* > <jay.lau....@gmail.com>> wrote: > > > > Thanks Peng, then I can see two integration points for Magnum and > Hyper: > > 1) Once Hyper and k8s integration finished, we can deploy k8s in > two mode: docker and hyper mode, the end user can select which mode they > want to use. For such case, we do not need to create a new bay but may > need > some enhancement for current k8s bay > > 2) After mesos and hyper integration, we can treat mesos and hyper > as a new bay to magnum. Just like what we are doing now for > mesos+marathon. > > Thanks! > > > > 2015-07-16 17:38 GMT+08:00 Peng Zhao <*p...@hyper.sh* > <p...@hyper.sh>>: > > > Hi Jay, > > Yes, we are working with the community to integrate Hyper with Mesos and > K8S. Since Hyper uses Pod as the default job unit, it is quite easy to > integrate with K8S. Mesos takes a bit more efforts, but still > straightforward. > > We expect to finish both integration in v0.4 early August. > > Best, > Peng > > ----------------------------------------------------- > Hyper - Make VM run like Container > > > > On Thu, Jul 16, 2015 at 3:47 PM, Jay Lau <*jay.lau....@gmail.com* > <jay.lau....@gmail.com>> wrote: > > > Hi Peng, > > Just want to get more for Hyper. If we create a hyper bay, then can I > set up multiple hosts in a hyper bay? If so, who will do the scheduling, > does mesos or some others integrate with hyper? > I did not find much info for hyper cluster management. > > Thanks. > > 2015-07-16 9:54 GMT+08:00 Peng Zhao <*p...@hyper.sh* <p...@hyper.sh>>: > > > > > > > > > > > ------------------ Original ------------------ > *From: * “Adrian Otto”<*adrian.o...@rackspace.com* > <adrian.o...@rackspace.com>>; > *Date: * Wed, Jul 15, 2015 02:31 AM > *To: * “OpenStack Development Mailing List (not for usage questions)“< > *openstack-dev@lists.openstack.org* <openstack-dev@lists.openstack.org>>; > > > *Subject: * Re: [openstack-dev] [magnum][bp] Power Magnum to run > onmetal withHyper > > Peng, > > On Jul 13, 2015, at 8:37 PM, Peng Zhao <*p...@hyper.sh* > <p...@hyper.sh>> wrote: > > Thanks Adrian! > > Hi, all, > > Let me recap what is hyper and the idea of hyperstack. > > Hyper is a single-host runtime engine. Technically, > Docker = LXC + AUFS > Hyper = Hypervisor + AUFS > where AUFS is the Docker image. > > I do not understand the last line above. My understanding is that AUFS > == UnionFS, which is used to implement a storage driver for Docker. Others > exist for btrfs, and devicemapper. You select which one you want by setting > an option like this: > > DOCKEROPTS=”-s devicemapper” > > Are you trying to say that with Hyper, AUFS is used to provide layered > Docker image capability that are shared by multiple hypervisor guests? > > > Peng >>> Yes, AUFS implies the Docker images here. > > > My guess is that you are trying to articulate that a host running > Hyper is a 1:1 substitute for a host running Docker, and will respond using > the Docker remote API. This would result in containers running on the same > host that have a superior security isolation than they would if LXC was > used as the backend to Docker. Is this correct? > > > Peng>>> Exactly > > > Due to the shared-kernel nature of LXC, Docker lacks of the necessary > isolation in a multi-tenant CaaS platform, and this is what > Hyper/hypervisor is good at. > > And because of this, most CaaS today run on top of IaaS: > *https://trello-attachments.s3.amazonaws.com/55545e127c7cbe0ec5b82f2b/388x275/e286dea1266b46c1999d566b0f9e326b/iaas.png* > <https://trello-attachments.s3.amazonaws.com/55545e127c7cbe0ec5b82f2b/388x275/e286dea1266b46c1999d566b0f9e326b/iaas.png> > Hyper enables the native, secure, bare-metal CaaS > *https://trello-attachments.s3.amazonaws.com/55545e127c7cbe0ec5b82f2b/395x244/828ad577dafb3f357e95899e962651b2/caas.png* > <https://trello-attachments.s3.amazonaws.com/55545e127c7cbe0ec5b82f2b/395x244/828ad577dafb3f357e95899e962651b2/caas.png> > > From the tech stack perspective, Hyperstack turns Magnum o run in parallel > with Nova, not running on atop. > > For this to work, we’d expect to get a compute host from Heat, so if the > bay type were set to “hyper”, we’d need to use a template that can produce > a compute host running Hyper. How would that host be produced, if we do not > get it from nova? Might it make more sense to make a dirt driver for nova > that could produce a Hyper guest on a host already running the nova-compute > agent? That way Magnum would not need to re-create any of Nova’s > functionality in order to produce nova instances of type “hyper”. > > Peng >>> We don’t have to get the physical host from nova. Let’s say > OpenStack = Nova+Cinder+Neutron+Bare-metal+KVM, so “AWS-like IaaS for > everyone else” > HyperStack= Magnum+Cinder+Neutron+Bare-metal+Hyper, then “Google-like > CaaS for everyone else” > > Ideally, customers should deploy a single OpenStack cluster, with both > nova/kvm and magnum/hyper. I’m looking for a solution to make nova/magnum > co-exist. > > > Is Hyper compatible with libvirt? > > > Peng>>> We are working on the libvirt integration, expect in v0.5 > > > > Can Hyper support nested Docker containers within the Hyper guest? > > > Peng>>> Docker in Docker? In a HyperVM instance, there is no docker > daemon, cgroup and namespace (except MNT for pod). VM serves the purpose of > isolation. We plan to support cgroup and namespace, so you can control > whether multiple containers in a pod share the same namespace, or > completely isolated. But in either case, no docker daemon is present. > > > > Thanks, > > Adrian Otto > > > Best, > Peng > > ------------------ Original ------------------ > *From: * “Adrian Otto”<*adrian.o...@rackspace.com* > <adrian.o...@rackspace.com>>; > *Date: * Tue, Jul 14, 2015 07:18 AM > *To: * “OpenStack Development Mailing List (not for usage > questions)“<*openstack-dev@lists.openstack.org* > <openstack-dev@lists.openstack.org>>; > > *Subject: * Re: [openstack-dev] [magnum][bp] Power Magnum to run on > metal withHyper > > Team, > > I woud like to ask for your input about adding support for Hyper in > Magnum: > > *https://blueprints.launchpad.net/magnum/+spec/hyperstack* > <https://blueprints.launchpad.net/magnum/+spec/hyperstack> > > We touched on this in our last team meeting, and it was apparent > that achieving a higher level of understanding of the technology before > weighing in about the directional approval of this blueprint. Peng Zhao > and > Xu Wang have graciously agreed to respond to this thread to address > questions about how the technology works, and how it could be integrated > with Magnum. > > Please take a moment to review the blueprint, and ask your > questions here on this thread. > > Thanks, > > Adrian Otto > > On Jul 2, 2015, at 8:48 PM, Peng Zhao <*p...@hyper.sh* > <p...@hyper.sh>> wrote: > > > > Here is the bp of Magnum+Hyper+Metal integration: > *https://blueprints.launchpad.net/magnum/+spec/hyperstack* > <https://blueprints.launchpad.net/magnum/+spec/hyperstack> > > Wanted to hear more thoughts and kickstart some brainstorming. > > Thanks, > Peng > > ----------------------------------------------------- > Hyper - Make VM run like Container > > > > __________________________________________________________________________ > > > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > *openstack-dev-requ...@lists.openstack.org* > <openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe > *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev* > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: *openstack-dev-requ...@lists.openstack.org* > <openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe > *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev* > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > *openstack-dev-requ...@lists.openstack.org?subject:unsubscribe* > <http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe> > *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev* > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > -- > > Thanks, > Jay Lau (Guangya Liu) > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > *openstack-dev-requ...@lists.openstack.org?subject:unsubscribe* > <http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe> > *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev* > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > *openstack-dev-requ...@lists.openstack.org?subject:unsubscribe* > <http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe> > *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev* > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > -- > > Thanks, > > Jay Lau (Guangya Liu) > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: *openstack-dev-requ...@lists.openstack.org* > <openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe > *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev* > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > *openstack-dev-requ...@lists.openstack.org?subject:unsubscribe* > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev* > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > -- > Thanks, > > Jay Lau (Guangya Liu) > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Thanks, Jay Lau (Guangya Liu)
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev