Re: Summary of Re: Making QEMU easier for management tools and applications
On Wed, Feb 12, 2020 at 01:54:42PM +, Stefan Hajnoczi wrote: > On Mon, Feb 10, 2020 at 05:43:13PM +0100, Markus Armbruster wrote: > > Stefan Hajnoczi writes: > > > > > On Tue, Feb 4, 2020 at 3:54 PM Markus Armbruster > > > wrote: > > >> = Ways to provide machine-friendly initial configuration = > > >> > > >> Two ways to provide machine-friendly initial configuration on par with > > >> QMP have been proposed: > > >> > > >> 1. Extend QMP > > >> > > >>Machines use the CLI only to configure a QMP socket. The remainder > > >>of the CLI becomes human-only, with much relaxed compatibility rules. > > >> > > >> 2. QAPIfy the CLI > > >> > > >>Provide a machine-friendly CLI based on QAPI and JSON. The current > > >>CLI becomes human-only, with much relaxed compatibility rules. > > > > > > Do we keep the existing CLI around in both cases? I'm concerned that > > > we're still following the HMP/QMP approach, which has left QEMU with > > > the legacy HMP monitor that we still haven't removed. > > > > The "HMP is legacy" idea is relatively recent. > > > > I think having separate interfaces for humans and machines makes sense, > > we just need to give both the attention and care they need and deserve. > > > > I think a human-friendly monitor is has its use, but it should ideally > > be done differently than we do HMP now. > > > > Likewise, human-friendly initial configuration should exist, but it > > should ideally be done differently than we do HMP now. > > > > > I'm in favor of simplifying QEMU at the expense of an incompatible CLI > > > change in QEMU 6.0. > > > > I'm convinced the current CLI needs cleanup badly, and that means > > incompatible change. The question is how and when to change it. > > > > Here's how I'd like us to do it: > > > > 1. Create machine-friendly initial configuration interface separate from > >the existing CLI > > > >Doesn't mean it cannot be a CLI. > > > > 2. Develop it step by step to feature parity with existing CLI > > > >If we identify misfeatures we don't want anymore, we should > >immediately deprecate them in the existing CLI instead. > > > > 2. Transition machine users to this new interface > > > > 3. Declare the existing CLI to be like HMP: for humans, may change > >incompatibly > > > > 4. Clean up existing CLI step by step to wrap around the > >machine-friendly interface > > > >Whatever we deprecated in step 2 goes to the bit bucket instead. > > > >I'm open to replacing the existing CLI by a separate wrapper process > >instead. > > > >Capability to translate to the machine-friendly interface is > >desirable, so human users can easily transition to the > >machine-friendly interface when they run into a need to automate. > > > > The risk is of course that we fail at step 4 and remain stuck with the > > CLI mess we've made. > > Yes, QEMU does not have a good track record of successfully converting > to new APIs and then removing old code. > > My worry is that this effort will result in the addition of even more > code but we'll still be stuck with the old cruft (both in the user > visible interface and in the implementation). This is why I think any new CLI ought to be done in a new binary, not qemu-system-. I think it is an easier proposition to sell to people that this is a clean break if we make it a new binary. The mere fact the binary exists will make people curious about it. If we add new stuff to existing binaries, it is essentially invisible unless you look for it. Separate binaries would also make life better for documentation IMHO, as we can clearly distinguish legacy and modern in the docs. Indeed the new binary doc shoudl be completely separate, so when people learn about it, they're not distracted by legacy. This way, even if we don't delete qemu-system- for a long time, the new binary would not be polluted by the legacy cruft, even if it still exists in some internal places. Ideally the goal would be that QemuOpts be entirely missing from any code linked into the new binary. This will be challenging given some of the places QemuOpts embeds itself. Perhaps we can split some of the source files to isolate the QemuOpts usage. The block layer is the biggest challenge here. > But we won't get anywhere if we don't try :). This sounds like a > significant project and I wonder if others would be willing to help if > you can break down the tasks for them. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: Summary of Re: Making QEMU easier for management tools and applications
On Mon, Feb 10, 2020 at 05:43:13PM +0100, Markus Armbruster wrote: > Stefan Hajnoczi writes: > > > On Tue, Feb 4, 2020 at 3:54 PM Markus Armbruster wrote: > >> = Ways to provide machine-friendly initial configuration = > >> > >> Two ways to provide machine-friendly initial configuration on par with > >> QMP have been proposed: > >> > >> 1. Extend QMP > >> > >>Machines use the CLI only to configure a QMP socket. The remainder > >>of the CLI becomes human-only, with much relaxed compatibility rules. > >> > >> 2. QAPIfy the CLI > >> > >>Provide a machine-friendly CLI based on QAPI and JSON. The current > >>CLI becomes human-only, with much relaxed compatibility rules. > > > > Do we keep the existing CLI around in both cases? I'm concerned that > > we're still following the HMP/QMP approach, which has left QEMU with > > the legacy HMP monitor that we still haven't removed. > > The "HMP is legacy" idea is relatively recent. > > I think having separate interfaces for humans and machines makes sense, > we just need to give both the attention and care they need and deserve. > > I think a human-friendly monitor is has its use, but it should ideally > be done differently than we do HMP now. > > Likewise, human-friendly initial configuration should exist, but it > should ideally be done differently than we do HMP now. > > > I'm in favor of simplifying QEMU at the expense of an incompatible CLI > > change in QEMU 6.0. > > I'm convinced the current CLI needs cleanup badly, and that means > incompatible change. The question is how and when to change it. > > Here's how I'd like us to do it: > > 1. Create machine-friendly initial configuration interface separate from >the existing CLI > >Doesn't mean it cannot be a CLI. > > 2. Develop it step by step to feature parity with existing CLI > >If we identify misfeatures we don't want anymore, we should >immediately deprecate them in the existing CLI instead. > > 2. Transition machine users to this new interface > > 3. Declare the existing CLI to be like HMP: for humans, may change >incompatibly > > 4. Clean up existing CLI step by step to wrap around the >machine-friendly interface > >Whatever we deprecated in step 2 goes to the bit bucket instead. > >I'm open to replacing the existing CLI by a separate wrapper process >instead. > >Capability to translate to the machine-friendly interface is >desirable, so human users can easily transition to the >machine-friendly interface when they run into a need to automate. > > The risk is of course that we fail at step 4 and remain stuck with the > CLI mess we've made. Yes, QEMU does not have a good track record of successfully converting to new APIs and then removing old code. My worry is that this effort will result in the addition of even more code but we'll still be stuck with the old cruft (both in the user visible interface and in the implementation). But we won't get anywhere if we don't try :). This sounds like a significant project and I wonder if others would be willing to help if you can break down the tasks for them. Stefan signature.asc Description: PGP signature
Re: Summary of Re: Making QEMU easier for management tools and applications
Stefan Hajnoczi writes: > On Tue, Feb 4, 2020 at 3:54 PM Markus Armbruster wrote: >> = Ways to provide machine-friendly initial configuration = >> >> Two ways to provide machine-friendly initial configuration on par with >> QMP have been proposed: >> >> 1. Extend QMP >> >>Machines use the CLI only to configure a QMP socket. The remainder >>of the CLI becomes human-only, with much relaxed compatibility rules. >> >> 2. QAPIfy the CLI >> >>Provide a machine-friendly CLI based on QAPI and JSON. The current >>CLI becomes human-only, with much relaxed compatibility rules. > > Do we keep the existing CLI around in both cases? I'm concerned that > we're still following the HMP/QMP approach, which has left QEMU with > the legacy HMP monitor that we still haven't removed. The "HMP is legacy" idea is relatively recent. I think having separate interfaces for humans and machines makes sense, we just need to give both the attention and care they need and deserve. I think a human-friendly monitor is has its use, but it should ideally be done differently than we do HMP now. Likewise, human-friendly initial configuration should exist, but it should ideally be done differently than we do HMP now. > I'm in favor of simplifying QEMU at the expense of an incompatible CLI > change in QEMU 6.0. I'm convinced the current CLI needs cleanup badly, and that means incompatible change. The question is how and when to change it. Here's how I'd like us to do it: 1. Create machine-friendly initial configuration interface separate from the existing CLI Doesn't mean it cannot be a CLI. 2. Develop it step by step to feature parity with existing CLI If we identify misfeatures we don't want anymore, we should immediately deprecate them in the existing CLI instead. 2. Transition machine users to this new interface 3. Declare the existing CLI to be like HMP: for humans, may change incompatibly 4. Clean up existing CLI step by step to wrap around the machine-friendly interface Whatever we deprecated in step 2 goes to the bit bucket instead. I'm open to replacing the existing CLI by a separate wrapper process instead. Capability to translate to the machine-friendly interface is desirable, so human users can easily transition to the machine-friendly interface when they run into a need to automate. The risk is of course that we fail at step 4 and remain stuck with the CLI mess we've made. > A project like this could prototype incompatible CLI changes in a > separate git tree. If it achieves the desired unification (CLI, QMP, > configuration file) and simplification (less code, legacy removal) > then it can be merged for an upcoming QEMU major release. That's effectively a really long-lived feature branch. Painful. If it it what it takes, we do it. I hope the process I just sketched permits more incremental development. We can explore conflict-free in qemu-storage-daemon.
Re: Summary of Re: Making QEMU easier for management tools and applications
On Mon, 10 Feb 2020 at 11:08, Daniel P. Berrangé wrote: > > On Mon, Feb 10, 2020 at 11:01:48AM +, Peter Maydell wrote: > > On Mon, 10 Feb 2020 at 10:57, Stefan Hajnoczi wrote: > > > I'm in favor of simplifying QEMU at the expense of an incompatible CLI > > > change in QEMU 6.0. > > > > If we want to do wholesale incompatible changes to the CLI > > I think we definitely need some kind of tool where a user > > can say "here's my old command line, what's the new style > > equivalent?". Otherwise we're going to have a deluge of > > user issues where their old working setups broke and > > QEMU didn't give them any useful hints about why. > > There is a risk that if we promise to have a fully automated conversion > that it will be alot of work, and could force us to introduce hacks into > the new impl just to satisfy conversion. IMHO we shouldn't be afraid of > declaring that some parts of the old syntax can NOT be directly transformed > into new syntax, simply for the sake of making a new impl more practical > to move forward with. Agreed, but we should at least be able to handle the easy stuff and say "this is the general kind of new option syntax and set of options you want" for most of the rest. > An alternative approach to mitigate the disruption is to *not* make any > incompatible changes to qemu-system-. Instead introduce new binaries > with the new syntax and any other architectural changes we wish to make. > The old binaries can be deprecated but remain around for an extended > period of time, to give people and apps time to migrate. We can provide > rough guidance and perhaps partially automated conversion to help people > move, but not aim for a 100% automated conversion. I think our history of failing to actually complete transitions would predict that we'd end up with both the old and the new binaries essentially forever. thanks -- PMM
Re: Summary of Re: Making QEMU easier for management tools and applications
On Mon, Feb 10, 2020 at 11:01:48AM +, Peter Maydell wrote: > On Mon, 10 Feb 2020 at 10:57, Stefan Hajnoczi wrote: > > I'm in favor of simplifying QEMU at the expense of an incompatible CLI > > change in QEMU 6.0. > > If we want to do wholesale incompatible changes to the CLI > I think we definitely need some kind of tool where a user > can say "here's my old command line, what's the new style > equivalent?". Otherwise we're going to have a deluge of > user issues where their old working setups broke and > QEMU didn't give them any useful hints about why. There is a risk that if we promise to have a fully automated conversion that it will be alot of work, and could force us to introduce hacks into the new impl just to satisfy conversion. IMHO we shouldn't be afraid of declaring that some parts of the old syntax can NOT be directly transformed into new syntax, simply for the sake of making a new impl more practical to move forward with. An alternative approach to mitigate the disruption is to *not* make any incompatible changes to qemu-system-. Instead introduce new binaries with the new syntax and any other architectural changes we wish to make. The old binaries can be deprecated but remain around for an extended period of time, to give people and apps time to migrate. We can provide rough guidance and perhaps partially automated conversion to help people move, but not aim for a 100% automated conversion. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: Summary of Re: Making QEMU easier for management tools and applications
On 10/02/20 11:56, Stefan Hajnoczi wrote: > On Tue, Feb 4, 2020 at 3:54 PM Markus Armbruster wrote: >> = Ways to provide machine-friendly initial configuration = >> >> Two ways to provide machine-friendly initial configuration on par with >> QMP have been proposed: >> >> 1. Extend QMP >> >>Machines use the CLI only to configure a QMP socket. The remainder >>of the CLI becomes human-only, with much relaxed compatibility rules. >> >> 2. QAPIfy the CLI >> >>Provide a machine-friendly CLI based on QAPI and JSON. The current >>CLI becomes human-only, with much relaxed compatibility rules. > > Do we keep the existing CLI around in both cases? I'm concerned that > we're still following the HMP/QMP approach, which has left QEMU with > the legacy HMP monitor that we still haven't removed. > > I'm in favor of simplifying QEMU at the expense of an incompatible CLI > change in QEMU 6.0. > > A project like this could prototype incompatible CLI changes in a > separate git tree. If it achieves the desired unification (CLI, QMP, > configuration file) and simplification (less code, legacy removal) > then it can be merged for an upcoming QEMU major release. I think Daniel had a good point in suggesting a (possibly) throwaway fork for either (1) or (2). Let's see what kind of change is needed to do 100% QMP-based configuration of guests (or at least to QMP-ify configuration of devices and backends---things that can already have an *-add command now); then we can figure out which subset of the current CLI can be mapped to it. Paolo
Re: Summary of Re: Making QEMU easier for management tools and applications
On Mon, 10 Feb 2020 at 10:57, Stefan Hajnoczi wrote: > I'm in favor of simplifying QEMU at the expense of an incompatible CLI > change in QEMU 6.0. If we want to do wholesale incompatible changes to the CLI I think we definitely need some kind of tool where a user can say "here's my old command line, what's the new style equivalent?". Otherwise we're going to have a deluge of user issues where their old working setups broke and QEMU didn't give them any useful hints about why. thanks -- PMM
Re: Summary of Re: Making QEMU easier for management tools and applications
On Tue, Feb 4, 2020 at 3:54 PM Markus Armbruster wrote: > = Ways to provide machine-friendly initial configuration = > > Two ways to provide machine-friendly initial configuration on par with > QMP have been proposed: > > 1. Extend QMP > >Machines use the CLI only to configure a QMP socket. The remainder >of the CLI becomes human-only, with much relaxed compatibility rules. > > 2. QAPIfy the CLI > >Provide a machine-friendly CLI based on QAPI and JSON. The current >CLI becomes human-only, with much relaxed compatibility rules. Do we keep the existing CLI around in both cases? I'm concerned that we're still following the HMP/QMP approach, which has left QEMU with the legacy HMP monitor that we still haven't removed. I'm in favor of simplifying QEMU at the expense of an incompatible CLI change in QEMU 6.0. A project like this could prototype incompatible CLI changes in a separate git tree. If it achieves the desired unification (CLI, QMP, configuration file) and simplification (less code, legacy removal) then it can be merged for an upcoming QEMU major release. Stefan
Re: Summary of Re: Making QEMU easier for management tools and applications
Markus Armbruster writes: [...] > = Ways to provide machine-friendly initial configuration = > > Two ways to provide machine-friendly initial configuration on par with > QMP have been proposed: > > 1. Extend QMP > >Machines use the CLI only to configure a QMP socket. The remainder >of the CLI becomes human-only, with much relaxed compatibility rules. > > 2. QAPIfy the CLI > >Provide a machine-friendly CLI based on QAPI and JSON. The current >CLI becomes human-only, with much relaxed compatibility rules. > >Aside: I looked into cleaning up the human-only CLI at the same time, >but the need to maintain compatibility until the transition to the >machine-friendly CLI is complete makes this hard. It needs to be >cleaned up, though. More on that below. Forgot to write down: we're talking not just about qemu-system-FOO, but any executable with a non-trivial command line for use by machines. This includes qemu-img, future qemu-storage-daemon, possibly qemu-nbd and more. Of these, qemu-storage-daemon will have a QMP monitor. The others don't have one so far. [...]
Summary of Re: Making QEMU easier for management tools and applications
This thread has accumulated more than a hundred messages over seven weeks. We need a summary. The conversation has gone in several directions. In this message, I'll cover just the one I consider most important: machine-friendly initial configuration. I'll do the rest later, if I have stamina left. = QMP is fine for machines, CLI is not = We have two major external host interfaces: the CLI and QMP. Compare: QMP * Purpose: control at run-time * Commands with argument and return value * Events with argument value * Simple type system * Based on QAPI * Regular, well-defined syntax: QAPI schema on top of JSON * Documentation is required; actual documentation is of mixed quality * Introspectable, except for a few mostly QOM-related holes where we bypass QAPI * Machine-friendly, complemented by human-only HMP CLI * Purpose: initial configuration * Options with argument * Based on a crazy zoo of QemuOpts (with and without dotted keys), QAPI (only a few recent options), ad hoc parsers * Option argument syntax is mostly (variations of) key=value,... * QemuOpts' "type system" is "list of key-value pairs, where value can be string, bool, (unsigned) integer, or size", optionally restricted to known keys with known value types. * QAPI type system is the same as in QMP * Introspection is completely inadequate: misses options, incorrect option names, misses option arguments partly or completely * Configuration files with INI-like syntax, completely inadequate: can't do most options * Not machine-friendly * Maintaining it is a pain, evolving it is worse The inadequacy of the CLI has become a serious issue. = Ways to provide machine-friendly initial configuration = Two ways to provide machine-friendly initial configuration on par with QMP have been proposed: 1. Extend QMP Machines use the CLI only to configure a QMP socket. The remainder of the CLI becomes human-only, with much relaxed compatibility rules. 2. QAPIfy the CLI Provide a machine-friendly CLI based on QAPI and JSON. The current CLI becomes human-only, with much relaxed compatibility rules. Aside: I looked into cleaning up the human-only CLI at the same time, but the need to maintain compatibility until the transition to the machine-friendly CLI is complete makes this hard. It needs to be cleaned up, though. More on that below. To extend QMP, we wrap QMP commands around the internal initial configuration interfaces. These QMP commands take arguments, but don't return anything. Many of them will only make sense during initial configuration. We'll want to express that in the schema, and enforce it in the QMP core. Others will behave differently, e.g. cold plug during initial configuration, hot plug once the guest runs. Configuration files are just QMP commands from the initial configuration subset read from a file instead of a chardev. JSON is a poor choice for configuration files, and QMP's verbosity makes it poorer. We'll want more suitable concrete syntax for configuration files. To improve the CLI, we wrap QAPI-based CLI options around the internal initial configuration interfaces. We'll want some infrastructure to generate CLI option boilerplate, just like we generate QMP command boilerplate. Configuration files are just CLI options read from a file instead of argv[]. Again, we'll want a more suitable concrete syntax there. With an improved CLI, I'd expect machines to use configuration files so they don't have to mess with shell quoting. With extended QMP, they'd perhaps rather reuse their existing QMP code to send the configuration down a socket. Less efficient, but I doubt it'll matter. In both cases, the internal configuration interfaces need to be converted from QemuOpts to QAPI types. The one clear advantage of extending QMP is the ability to mix query commands with initial configuration. This is also a clear disadvantage: we need to make it work. Existing QMP commands may rely on initial configuration to be complete. We may have to allow only QMP commands we carefully checked. How important is the advantage? = Cleaning up the human-only CLI = In both cases, the existing CLI remains in a human-only role. That's a truckload of messy code to maintain. Not good. Can we replace it by thin wrappers around the machine-friendly alternative, ideally incrementally? Can we generate a useful part of these wrappers? If we QAPIfy the CLI, the QAPI generator already knows the CLI, and only has to be taught the general human-friendly key=value,... syntax, plus a way to specify exceptions. If we extend QMP, we'll additionally need some of the CLI QAPIfication infrastructure for this. Hmm.
