* Kevin Wolf (kw...@redhat.com) wrote: > Am 06.02.2020 um 10:40 hat Markus Armbruster geschrieben: > > >> On 2/5/20 8:09 AM, Kevin Wolf wrote: > > >> > Am 28.01.2020 um 11:59 hat Kevin Wolf geschrieben: > > >> >>>> The other part that it needs to solve is how to be available by > > >> >>>> default > > >> >>>> without specifying anything on the command line. Basically, if I > > >> >>>> press > > >> >>>> Ctrl-Alt-2, I want to get to a monitor shell. If that shell is > > >> >>>> implemented internally or by an external Python process, I don't > > >> >>>> mind. > > >> >>> > > >> >>> That is a harder part. (I rarely use Ctrl-Alt-2 actually; I mostly > > >> >>> use HMP on stdin). > > >> >> > > >> >> I don't think it would be that hard, actually. > > >> >> > > >> >> If you have a -qmp-shell option that takes a chardev and defaults to > > >> >> vc, > > >> >> you've solved the part with both stdio and Ctrl-Alt-2. Now all you > > >> >> need > > >> >> to do is launch the Python child process, pass it a pair of pipes for > > >> >> communication and forward everything between the pipes and the > > >> >> chardev. > > >> >> > > >> >> (That's the theory anyway.) > > >> > > > >> > If someone is interested, I did a quick proof-of-concept hack: > > >> > > > >> > https://repo.or.cz/qemu/kevin.git/shortlog/refs/heads/qmp-shell > > >> > > > >> > It doesn't clean up anything properly (including the qmp-shell > > >> > processes > > >> > it starts), but it spawns a usable qmp-shell on a user-specified > > >> > character device. stdio seems to work, though without readline > > >> > functionality (I suppose I still have line-buffering somewhere), vc > > >> > doesn't really work at all yet. > > >> > > > >> > Try it out like this: > > >> > > > >> > $ ./qemu-storage-daemon --chardev stdio,id=m --monitor > > >> > m,mode=qmp-shell > > >> > monitor_qmp_event: 1 > > >> > Welcome to the QMP low-level shell! > > >> > Connected to QEMU 4.2.50 > > >> > > > >> > (QEMU) query-version > > >> > {"return": {"qemu": {"micro": 50, "major": 4, "minor": 2}, > > >> > "package": "v4.2.0-1188-gd95a3885a9"}} > > >> > (QEMU) quit > > >> > > > >> > (Or use x86_64-softmmu/qemu-system-x86_64, but it's based on the > > >> > refactorings in the storage daemon branch, so why not try both at > > >> > once?) > > >> > > > >> > Polishing this to make it mergable would still require substantial > > >> > work, > > >> > so at the moment I'm not planning to do this. But if someone wants to > > >> > pick it up, feel free (just let us know). > > >> > > > >> > Hm, in fact... A qmp-shell GSoC project? > > >> > > > >> > > >> That would be great. I worry that we should have a clear vision for the > > >> syntax before we give this project to an intern, though. With a clear > > >> vision and an outline for deliverables, it's an incredibly appropriate > > >> project. > > >> > > >> Some things I think we want to define before we start: > > >> > > >> 1. What are we trying to achieve with a standalone shell? > > > > Projects without a clear goal rarely succeed. Success within three > > months is even rarer. > > > > >> 2. What syntax should it use? > > > > Leaving that to a GSoC student amounts to setting up for failure. > > I think this subthread shows that we actually have many separate > projects that people wish to have someone work on. Each of them is > probably a bit too small for a whole GSoC, but all of them together are > probably too much. So I'll guess the student would pick maybe two of > them, and if time is left at the end, more can be added as a bonus. > > 1. Something like --monitor mode=qmp-shell that just spawns an external > Python script and passes it a QMP socket. This is the fundamental > building block for having any kind of external monitor script > actually integrated in QEMU, so I think just running the existing > qmp-shell this way (with proper support for at least stdio and vc > chardevs) would make sense as a first milestone.
I was originally going to suggest that should be sugar for a -chardev filter that takes an in/out chardev - but I don't know how you'd handle tty stuff like formatting and tab completion etc. > 2. Rewriting qmp-shell to use a better syntax for nested data > structures. This would have to be defined before the project starts. > > 3. Improving qmp-shell UI-wise, e.g. by having better autocompletion, > support for counting brackets, or whatever else was mentioned. We > have a few ideas, and there's room for the student to add their own > ideas, too. > > 4. Something HMP-like. This isn't QMP any more, so it could as well be a > separate script (hmp-shell?). But it could also be integrated in > qmp-shell in the form of additional commands that are implemented > client-side. Or maybe have a single shell, but have a QMP mode and an > HMP mode and the user can switch between these modes. > > The syntax for the HMP shell/mode could be the same or different from > the QMP syntax. This would have to be defined beforehand, too. separate script sharing the qmp interface code? Dave > 5. Probably more that I just forgot now. > > Suggesting the exact goals is part of the student application process, > but for fundamental things like the syntax we should probably already > know what we want. > > > >> I think those are the hardest parts. > > >> > > >> Below, some musings: > > >> > > >> - An integrated QMP shell would be a great usability boost to users of > > >> bare QEMU. > > >> > > >> - It is undesirable in general to support two interfaces. Feature > > >> disparity is a problem, as is needing to document and test two separate > > >> interfaces. The quality disparity between the two is also an issue. > > >> > > >> - Offering HMP via the GTK interface but not QMP is a discoverability > > >> problem. Unfamiliar users might assume that HMP is our flagship > > >> interface. It is not. > > >> > > >> - We are unlikely to re-expand HMP to cover everything QMP does; writing > > >> a QMP shell that makes QMP easy to interface with is a better solution > > >> for removing redundancy and complexity. > > I'm not entirely convinced of this because QMP is often too low-level to > actually address the practical high-level needs of users. > > But these HMP-ish things are probably easier to maintain as scripts > outside of the QEMU binary, so I think some kind of "QMP with > extensions" for human could be the solution. > > Once it's an external script, it will also be easy to exchange the shell > for another one depending on user preference, or to hack in whatever > functionality they are missing. > > > >> - I suspect that the target audience for users of naked QEMU are: > > >> - QEMU developers > > >> - Upper-layer developers (RHV, oVirt, KubeVirt, libvirt, kata, et al) > > >> researching, testing, and debugging integration. > > >> - Devops professionals testing, implementing and debugging > > >> configuration & infrastructure > > >> - Security/infosec researchers > > >> - Embedded platform developers > > >> - Academic researchers > > Maybe kernel developers should be mentioned separately, but yes, this > list looks plausible to me. > > > >> So please correct me if I am off the mark; > > >> > > >> Design Goals: > > >> - The removal of HMP > > >> - An easy-to-use interface that remains reasonably "close" to the > > >> machine API such that it provides a smooth transition to scripting QEMU. > > >> - Integration with our GTK interface for discoverability and > > >> convenience > > As I listed above, I think these are actually three separate projects, > rather than goals for a single big projects. > > > >> Syntax: > > >> - TBD? Do we agree that the current syntax in qmp-shell is "bad" and > > >> should be replaced? If yes, what should it look like? > > > > > > I believe it should be a python shell with added commands. > > > > > > Simple things should be simple. > > > e.g. adding a disk from a local file should be trivial. > > > > > > Complex things can be complex - but it would be better if they were > > > simple. > > > > > > It's OK if the worst case of a blockdev is a bit hairy, but > > > watch out for cases where the hairyness creeps in unnecessarily. > > > > Designing interfaces to complex machinery is hard. Experience tells > > that we do okay when we focus on the building blocks first. That's > > -blockdev. When we start with trying to make simple things simple, we > > end in swamps. That's -drive. > > > > Focus on building blocks is of course no excuse for unnecessary > > hairiness. > > > > It's also no reason not to build more convenient things on top of the > > building blocks. I doubt they should go into QMP, though. > > Right, they should be implemented in that external script, which would > use the lower-level QMP building blocks to provide the functionality. I > also think it's a good idea to keep QMP accessible for more exotic use > cases when the simple thing just doesn't cut it any more. > > > > If the user screwsup, it should give an error that prompts the user > > > to the parameter they got wrong. > > > > > > Output from commands should normally be pretty formatted (with an option > > > to display raw json for those needing it). > > > e.g. that 'query-version' should give either just the package > > > version (as info version currently does) or: > > > 4.2.50 Package: v4.2.0-1188-gd95a3885a9 > > > > > > We shouldn't lose any HMP commands that some people find useful > > > Ditching HMP isn't an option until we've got almost all of it > > > covered. > > > > In particular, we currently use HMP for debugging and monitoring > > purposes, where we don't need or want QMP's rigor, neither its rigorous > > interface stability, nor its structured I/O. We want the "whipuptitude" > > we get from monitor_printf(). This is actually a point David has made > > several times. > > > > To have a qmp-shell replace HMP, I think it needs to be able to > > > > * Go beyond 1:1 > > > > We tried a 1:1 mapping between HMP and QMP commands, and it didn't > > work out. HMP's replacement should let us build convenient commands > > from QMP building blocks. > > > > We tried a 1:1 mapping between HMP and QMP command arguments, guided > > by @args_type. Worked out for simple cases, but was too constricting. > > We need to go beyond 1:1, but we probably want to be able to offer 1:1 > as a subset of commands accepted in that shell. > > Offering only 1:1 in a good way might already be a step forward. > > > * Preserve "whipuptitude" [David] > > > > I figure that means allowing some in QMP. Without compromising its > > core mission, of course. > > As long as we confine it to x- commands, I think this is okay. > > > * As discoverable as HMP is now [Kevin] > > > > * Help, completion and such at least on par with what HMP provides now > > Will we want to add new annotations in the schema for this? > > For example, HMP has completion support for block device names. In the > QAPI schema, these are simply 'str'. We could bake the knowledge that > in command 'foo' the parameter 'bar' is a block device name, but that > would be a hack and would probably rarely be consistent with what QEMU > actually does. It's really something that schema introspection should be > able to tell us. > > Kevin -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK