[resending to include OSP devs ] YANIV LAVI
SENIOR TECHNICAL PRODUCT MANAGER Red Hat Israel Ltd. <https://www.redhat.com/> 34 Jerusalem Road, Building A, 1st floor Ra'anana, Israel 4350109 yl...@redhat.com T: +972-9-7692306/8272306 F: +972-9-7692223 IM: ylavi <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> @redhatnews <https://twitter.com/redhatnews> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> On Wed, Apr 4, 2018 at 7:23 PM, Yaniv Lavi <yl...@redhat.com> wrote: > [resending to include KubeVirt devs ] > > YANIV LAVI > > SENIOR TECHNICAL PRODUCT MANAGER > > Red Hat Israel Ltd. <https://www.redhat.com/> > > 34 Jerusalem Road, Building A, 1st floor > > Ra'anana, Israel 4350109 > > yl...@redhat.com T: +972-9-7692306/8272306 F: +972-9-7692223 IM: > ylavi > <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > @redhatnews <https://twitter.com/redhatnews> Red Hat > <https://www.linkedin.com/company/red-hat> Red Hat > <https://www.facebook.com/RedHatInc> > > > On Wed, Apr 4, 2018 at 7:07 PM, Yaniv Lavi <yl...@redhat.com> wrote: > >> Hi, >> I'd like to go one step back and discuss why we should try to do this on >> the high level. >> >> For the last 5-10 years of KVM development, we are pragmatically >> providing the Linux host level APIs via project specific host >> agents/integration code (Nova agent, oVirt host agent, virt-manager). >> In recent time we see new projects that have similar requirements >> (Cockpit, different automation tool, KubeVirt), this means that all of the >> Linux virt stack consumers are reinventing the wheel and using very >> different paths to consume the partial solutions that are provided today. >> >> The use of the Linux virt stack is well defined by the existing projects >> scope and it makes a lot of sense to try to provide the common patterns via >> the virt stack directly as a host level API that different client or >> management consume. >> The main goal is to improve the developer experience for virtualization >> management applications with an API set that is useful to the entire set of >> tools (OSP, oVirt, KubeVirt, Cockpit and so on). >> >> The Linux virt developer community currently is not able to provide best >> practices and optimizations from single node knowledge. This means that all >> of that smarts is locked to the specific project integration in the good >> case or not provided at all and the projects as a whole lose from that. >> When testing the Linux virt stack itself and since each project has >> different usage pattern, we lose the ability to test abilities on the lower >> level making the entire stack less stable and complete for new features. >> >> This also limits the different projects ability to contribute back to the >> Linux stack based on their user and market experience for others in open >> source to gain. >> >> I understand this shift is technically challenging for existing projects, >> but I do see value in doing this even for new implementation like Cockpit >> and KubeVirt. >> I also believe that the end result could be appealing enough to cause >> project like OSP, virt-manager and oVirt to consider to reduce the existing >> capabilities of their host side integrations/agents to shims on the host >> level and reuse the common/better-tested pattern as clients that was >> developed against the experience of the different projects. >> >> I call us all to collaborate and try to converge on a solution that will >> help all in the long term in the value you get from the common base. >> >> >> Thanks, >> >> YANIV LAVI >> >> SENIOR TECHNICAL PRODUCT MANAGER >> >> Red Hat Israel Ltd. <https://www.redhat.com/> >> >> 34 Jerusalem Road, Building A, 1st floor >> >> Ra'anana, Israel 4350109 >> >> yl...@redhat.com T: +972-9-7692306/8272306 F: +972-9-7692223 IM: >> ylavi >> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> >> @redhatnews <https://twitter.com/redhatnews> Red Hat >> <https://www.linkedin.com/company/red-hat> Red Hat >> <https://www.facebook.com/RedHatInc> >> >> >> On Thu, Mar 22, 2018 at 7:18 PM, Daniel P. Berrangé <berra...@redhat.com> >> wrote: >> >>> On Thu, Mar 22, 2018 at 03:54:01PM +0100, Martin Kletzander wrote: >>> > > >>> > > > One more thing could be automatically figuring out best values >>> based on >>> > > > libosinfo-provided data. >>> > > > >>> > > > 2) Policies >>> > > > >>> > > > Lot of the time there are parts of the domain definition that need >>> to be >>> > > > added, but nobody really cares about them. Sometimes it's enough >>> to >>> > > > have few templates, another time you might want to have a policy >>> > > > per-scenario and want to combine them in various ways. For >>> example with >>> > > > the data provided by point 1). >>> > > > >>> > > > For example if you want PCI-Express, you need the q35 machine >>> type, but >>> > > > you don't really want to care about the machine type. Or you want >>> to >>> > > > use SPICE, but you don't want to care about adding QXL. >>> > > > >>> > > > What if some of these policies could be specified once (using some >>> DSL >>> > > > for example), and used by virtuned to merge them in a unified and >>> > > > predictable way? >>> > > > >>> > > > 3) Abstracting the XML >>> > > > >>> > > > This is probably just usable for stateless apps, but it might >>> happen >>> > > > that some apps don't really want to care about the XML at all. >>> They >>> > > > just want an abstract view of the domain, possibly add/remove a >>> device >>> > > > and that's it. We could do that as well. I can't really tell how >>> much >>> > > > of a demand there is for it, though. >>> > > >>> > > It is safe to say that applications do not want to touch XML at all. >>> > > Any non-trivial application has created an abstraction around XML, >>> > > so that they have an API to express what they want, rather than >>> > > manipulating of strings to format/parse XML. >>> > > >>> > >>> > Sure, this was just meant to be a question as to whether it's worth >>> > pursuing or not. You make a good point on why it is not (at least for >>> > existing apps). >>> > >>> > However, since this was optional, the way this would look without the >>> > XML abstraction is that both input and output would be valid domain >>> > definitions, ultimately resulting in something similar to virt-xml with >>> > the added benefit of applying a policy from a file/string either >>> > supplied by the application itself. Whether that policy was taken from >>> > a common repository of such knowledge is orthogonal to this idea. >>> Since >>> > you would work with the same data, the upgrade could be incremental as >>> > you'd only let virtuned fill in values for new options and could slowly >>> > move on to using it for some pre-existing ones. None of the previous >>> > approaches did this, if I'm not mistaken. Of course it gets more >>> > difficult when you need to expose all the bits libvirt does and keep >>> > them in sync (as you write below). >>> >>> That has implications for how mgmt app deals with XML. Nova has object >>> models for representing XML in memory, but it doesn't aim to have >>> loss-less roundtrip from parse -> object -> format. So if Nova gets >>> basic XML from virttuned, parses it into its object to let it set >>> more fields and then formats it again, chances are it will have lost >>> a bunch of stuff from virttuned. Of course if you know about this >>> need upfront you can design the application such that it can safely >>> round-trip, but this is just example of problem with integrating to >>> existing apps. >>> >>> The other thing that concerns is that there are dependancies between >>> different bits of XML for a given device. ie if feature X is set to >>> a certain value, that prevents use of feature Y. So if virttuned >>> sets feature X, but the downstream application uses feature Y, the >>> final result can be incompatible. The application won't know this >>> because it doesn't control what stuff virttuned would be setting. >>> This can in turn cause ordering constraints. >>> >>> eg the application needs to say that virtio-net is being used, then >>> virttuned can set some defaults like enabling vhost-net, and then >>> the application can fill in more bits that it cares about. Or if >>> we let virttuned go first, setting virtio-net model + vhost-net, >>> then application wants to change model to e1000e, it has to be >>> aware that it must now delete the vhost-net bit that virtuned >>> added. This ends up being more complicated that just ignoring >>> virttuned and coding up use of vhost-net in application code. >>> >>> >>> > > This is the same kind of problem we faced wrt libvirt-gconfig and >>> > > libvirt-gobject usage from virt-manager - it has an extensive code >>> > > base that already works, and rewriting it to use something new >>> > > is alot of work for no short-term benefit. libvirt-gconfig/gobject >>> > > were supposed to be the "easy" bits for virt-manager to adopt, as >>> > > they don't really include much logic that would step on >>> virt-manager's >>> > > toes. libvirt-designer was going to be a very opinionated library >>> > > and in retrospective that makes it even harder to consider adopting >>> > > it for usage in virt-manager, as it'll have signficant liklihood >>> > > of making functionally significant changes in behaviour. >>> > > >>> > >>> > The initial idea (which I forgot to mention) was that all the decisions >>> > libvirt currently does (so that it keeps the guest ABI stable) would be >>> > moved into data (let's say some DSL) and it could then be switched or >>> > adjusted if that's not what the mgmt app wants (on a per-definition >>> > basis, of course). I didn't feel very optimistic about the upstream >>> > acceptance for that idea, so I figured that there could be something >>> > that lives beside libvirt, helps with some policies if requested and >>> > then the resulting XML could be fed into libvirt for determining the >>> > rest. >>> >>> I can't even imagine how we would go about encoding the stable guest >>> ABI logic libvirt does today in data ! >>> >>> > >>> > > There's also the problem with use of native libraries that would >>> > > impact many apps. We only got OpenStack to grudgingly allow the >>> > >>> > By native you mean actual binary libraries or native to the OpenStack >>> > code as in python module? Because what I had in mind for this project >>> > was a python module with optional wrapper for REST API. >>> >>> I meant native binary libraries. ie openstack is not happy in general >>> with adding dependancies on new OS services, because there's a big >>> time lag for getting them into all distros. By comparison a pure >>> python library, they can just handle automatically in their deployment >>> tools, just pip installing on any OS distro straight from pypi. This >>> is what made use of libosinfo a hard sell in Nova. >>> >>> The same thing is seen with Go / Rust where some applications have >>> decided they're better of actually re-implementing the libvirt RPC >>> protocol in Go / Rust rather than use the libvirt.so client. I think >>> this is a bad tradeoff in general, but I can see why they like it >>> >>> Regards, >>> Daniel >>> -- >>> |: https://berrange.com -o- https://www.flickr.com/photos/ >>> dberrange :| >>> |: https://libvirt.org -o- >>> https://fstop138.berrange.com :| >>> |: https://entangle-photo.org -o- https://www.instagram.com/dber >>> range :| >>> _______________________________________________ >>> Devel mailing list >>> Devel@ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel