Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/28/2011 06:01 AM, Anthony Liguori wrote: It would be a pity to divorce the monitor from chardevs, they're really flexible. Couple considerations: 1) chardevs don't support multiple simultaneous connections. I view this as a blocker for QMP. What do you mean by that? Something like ,server which keeps on listening after it a connection is established? ,server won't allow multiple simultaneous connections. CharDriverStates simply don't have a connection semantic. There can only be one thing connected to it at a time. This is why we don't use CharDriverState for VNC. I meant an extension to ,server that keeps on listening. We should have another abstraction for connection based backend. I'll take a go at this when I'm ready to try to get those patches in. Shouldn't each new connection return a chardev? Just to be clear though, there is a CharDriverState version of the new QMP server. This would be a second option for creating a QMP server and it takes a different command line sytnax. 2) Because chardevs don't support multiple connections, we can't reasonably hook on things like connect/disconnect which means that fd's sent via SCM_RIGHTs have to be handled in a very special way. By going outside of the chardev layer, we can let fd's via SCM_RIGHTS queue up naturally and have getfd/setfd refer to the fd at the top of the queue. It makes it quite a bit easier to work with (I believe Daniel had actually requested this a while ago). I really don't follow... what's the connection between SCM_RIGHTS and multiple connections? Monitors have a single fd. That fd is associated with the monitor and lives beyond the length of the connection to the monitor (recall that chardevs don't have a notion of connection life cycle). This means if a management tool forgets to do a closefd on an unused fd, there's no easy way for QEMU to automatically clean that up. IOW, a crashed management tool == fd leak in QEMU. I guess we could add a close() event to chardev? -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/27/2011 05:33 AM, Avi Kivity wrote: On 02/24/2011 07:25 PM, Anthony Liguori wrote: Is it really necessary? What's blocking us from initializing chardevs early? Well We initialize all chardevs at once right now and what set of chardevs there are depends on the machine (by the way defaults are applied). You could initialize chardevs in two stages although that requires quite a bit of additional complexity. We could initialize chardevs on demand - that should resolve any dependencies? I think that potentially screws up the way -global works. There's some deep black magic involved in how -global, defaults, and device initialization interact. It would be a pity to divorce the monitor from chardevs, they're really flexible. Couple considerations: 1) chardevs don't support multiple simultaneous connections. I view this as a blocker for QMP. What do you mean by that? Something like ,server which keeps on listening after it a connection is established? ,server won't allow multiple simultaneous connections. CharDriverStates simply don't have a connection semantic. There can only be one thing connected to it at a time. This is why we don't use CharDriverState for VNC. We should have another abstraction for connection based backend. I'll take a go at this when I'm ready to try to get those patches in. Just to be clear though, there is a CharDriverState version of the new QMP server. This would be a second option for creating a QMP server and it takes a different command line sytnax. 2) Because chardevs don't support multiple connections, we can't reasonably hook on things like connect/disconnect which means that fd's sent via SCM_RIGHTs have to be handled in a very special way. By going outside of the chardev layer, we can let fd's via SCM_RIGHTS queue up naturally and have getfd/setfd refer to the fd at the top of the queue. It makes it quite a bit easier to work with (I believe Daniel had actually requested this a while ago). I really don't follow... what's the connection between SCM_RIGHTS and multiple connections? Monitors have a single fd. That fd is associated with the monitor and lives beyond the length of the connection to the monitor (recall that chardevs don't have a notion of connection life cycle). This means if a management tool forgets to do a closefd on an unused fd, there's no easy way for QEMU to automatically clean that up. IOW, a crashed management tool == fd leak in QEMU. (6) can be started right now. (1) comes with the QAPI merge. (2) is pretty easy to do after applying this patch. (3) is probably something that can be done shortly after (1). (4) and (5) really require everything but (6) to be in place before we can meaningful do it. I think we can lay out much of the ground work for this in 0.15 and I think we can have a total conversion realistically for 0.16. That means that by EOY, we could invoke QEMU with no options and do everything through QMP. It's something that I've agitated for a long while, but when I see all the work needed, I'm not sure it's cost effective. There's a lot of secondary benefits that come from doing this. QMP becomes a much stronger interface. A lot of operations that right now are only specifiable by the command line become dynamic which mitigates reboots in the long term. Only the hot-pluggable ones. Yup, but it forces us to treat options that cannot change at runtime as special cases which I think is a nice plus. Customers don't like having their guests rebooted during a scheduled downtime so we really ought to try to have as many things tunable at runtime as possible. Regards, Anthony Liguori
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/24/2011 07:25 PM, Anthony Liguori wrote: Is it really necessary? What's blocking us from initializing chardevs early? Well We initialize all chardevs at once right now and what set of chardevs there are depends on the machine (by the way defaults are applied). You could initialize chardevs in two stages although that requires quite a bit of additional complexity. We could initialize chardevs on demand - that should resolve any dependencies? It would be a pity to divorce the monitor from chardevs, they're really flexible. Couple considerations: 1) chardevs don't support multiple simultaneous connections. I view this as a blocker for QMP. What do you mean by that? Something like ,server which keeps on listening after it a connection is established? 2) Because chardevs don't support multiple connections, we can't reasonably hook on things like connect/disconnect which means that fd's sent via SCM_RIGHTs have to be handled in a very special way. By going outside of the chardev layer, we can let fd's via SCM_RIGHTS queue up naturally and have getfd/setfd refer to the fd at the top of the queue. It makes it quite a bit easier to work with (I believe Daniel had actually requested this a while ago). I really don't follow... what's the connection between SCM_RIGHTS and multiple connections? 3) By treating QMP as a special case, we don't have to treat chardevs overall as a special case. This feels more right to me although I can't say I have a strong opinion formed yet. 2) Make qemu_machine_init() take no parameters and just reference global state. 3) Teach all QMP functions to behave themselves if called before qemu_machine_init() 4) Introduce QMP function to call qemu_machine_init() An alternative is to remove all guest-visible content from qemu_machine_init(). So machine->init() would take no parameters and only build the static devices (power supply?). Everything else would be hot-plugged (perhaps some would fail if the machine was started - cold-plug only). All that qemu_machine_init() is is guest-visible content. That's the point of refactoring this. Sorry, poorly phrased. Configurable guest visible content. (6) can be started right now. (1) comes with the QAPI merge. (2) is pretty easy to do after applying this patch. (3) is probably something that can be done shortly after (1). (4) and (5) really require everything but (6) to be in place before we can meaningful do it. I think we can lay out much of the ground work for this in 0.15 and I think we can have a total conversion realistically for 0.16. That means that by EOY, we could invoke QEMU with no options and do everything through QMP. It's something that I've agitated for a long while, but when I see all the work needed, I'm not sure it's cost effective. There's a lot of secondary benefits that come from doing this. QMP becomes a much stronger interface. A lot of operations that right now are only specifiable by the command line become dynamic which mitigates reboots in the long term. Only the hot-pluggable ones. It also lays the ground work for a fully decoupled device model whereas the only interface between the devices and the outside world is a subset of QMP (think seccomp()). Whether creating a machine with no command line options is high value is probably irrelevant. I think we want to go in this direction regardless. I agree it's a good thing. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/24/2011 10:01 AM, Avi Kivity wrote: On 02/24/2011 01:12 AM, Anthony Liguori wrote: What is the plan from here? 1) Decouple QMP from qemu_machine_init(). This really requires the introduction of the new QAPI server that exists outside of the chardev infrastructure since chardevs are currently initialized in qemu_machine_init(). Is it really necessary? What's blocking us from initializing chardevs early? Well We initialize all chardevs at once right now and what set of chardevs there are depends on the machine (by the way defaults are applied). You could initialize chardevs in two stages although that requires quite a bit of additional complexity. It would be a pity to divorce the monitor from chardevs, they're really flexible. Couple considerations: 1) chardevs don't support multiple simultaneous connections. I view this as a blocker for QMP. 2) Because chardevs don't support multiple connections, we can't reasonably hook on things like connect/disconnect which means that fd's sent via SCM_RIGHTs have to be handled in a very special way. By going outside of the chardev layer, we can let fd's via SCM_RIGHTS queue up naturally and have getfd/setfd refer to the fd at the top of the queue. It makes it quite a bit easier to work with (I believe Daniel had actually requested this a while ago). 3) By treating QMP as a special case, we don't have to treat chardevs overall as a special case. This feels more right to me although I can't say I have a strong opinion formed yet. 2) Make qemu_machine_init() take no parameters and just reference global state. 3) Teach all QMP functions to behave themselves if called before qemu_machine_init() 4) Introduce QMP function to call qemu_machine_init() An alternative is to remove all guest-visible content from qemu_machine_init(). So machine->init() would take no parameters and only build the static devices (power supply?). Everything else would be hot-plugged (perhaps some would fail if the machine was started - cold-plug only). All that qemu_machine_init() is is guest-visible content. That's the point of refactoring this. 5) Introduce new command line flag to not automatically call qemu_machine_init() 6) Convert all command line options to just be QMP function calls (6) can be started right now. (1) comes with the QAPI merge. (2) is pretty easy to do after applying this patch. (3) is probably something that can be done shortly after (1). (4) and (5) really require everything but (6) to be in place before we can meaningful do it. I think we can lay out much of the ground work for this in 0.15 and I think we can have a total conversion realistically for 0.16. That means that by EOY, we could invoke QEMU with no options and do everything through QMP. It's something that I've agitated for a long while, but when I see all the work needed, I'm not sure it's cost effective. There's a lot of secondary benefits that come from doing this. QMP becomes a much stronger interface. A lot of operations that right now are only specifiable by the command line become dynamic which mitigates reboots in the long term. It also lays the ground work for a fully decoupled device model whereas the only interface between the devices and the outside world is a subset of QMP (think seccomp()). Whether creating a machine with no command line options is high value is probably irrelevant. I think we want to go in this direction regardless. Regards, Anthony Liguori
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/24/2011 01:12 AM, Anthony Liguori wrote: What is the plan from here? 1) Decouple QMP from qemu_machine_init(). This really requires the introduction of the new QAPI server that exists outside of the chardev infrastructure since chardevs are currently initialized in qemu_machine_init(). Is it really necessary? What's blocking us from initializing chardevs early? It would be a pity to divorce the monitor from chardevs, they're really flexible. 2) Make qemu_machine_init() take no parameters and just reference global state. 3) Teach all QMP functions to behave themselves if called before qemu_machine_init() 4) Introduce QMP function to call qemu_machine_init() An alternative is to remove all guest-visible content from qemu_machine_init(). So machine->init() would take no parameters and only build the static devices (power supply?). Everything else would be hot-plugged (perhaps some would fail if the machine was started - cold-plug only). 5) Introduce new command line flag to not automatically call qemu_machine_init() 6) Convert all command line options to just be QMP function calls (6) can be started right now. (1) comes with the QAPI merge. (2) is pretty easy to do after applying this patch. (3) is probably something that can be done shortly after (1). (4) and (5) really require everything but (6) to be in place before we can meaningful do it. I think we can lay out much of the ground work for this in 0.15 and I think we can have a total conversion realistically for 0.16. That means that by EOY, we could invoke QEMU with no options and do everything through QMP. It's something that I've agitated for a long while, but when I see all the work needed, I'm not sure it's cost effective. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/24/2011 04:19 AM, Stefan Hajnoczi wrote: Any chance of reusing info qtree, QemuOpts, or other existing infrastructure for the config file? I'm nowhere near implementation details like that. I'm still trying to understand whether this is a Good Idea at all. Regards, Anthony Liguori Config file set/get/format code tends to be just toggling variables and manipulating strings. There is similarity here with the option parsing and qdev properties. Stefan
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On Thu, Feb 24, 2011 at 12:36 AM, Anthony Liguori wrote: > On 02/23/2011 05:38 PM, Juan Quintela wrote: >> >> Anthony Liguori wrote: >> >>> >>> On 02/23/2011 05:00 PM, Juan Quintela wrote: >>> Anthony Liguori wrote: > > The goal is to enable the monitor to run independently of whether the > machine > has been created such that the monitor can be used to specify all of > the > parameters for machine initialization. > > Signed-off-by: Anthony Liguori > > I agree that it is one step in the right direction, but we are still calling qemu_machine_init() before calling the main_loop(). What is the plan from here? >>> >>> 1) Decouple QMP from qemu_machine_init(). This really requires the >>> introduction of the new QAPI server that exists outside of the chardev >>> infrastructure since chardevs are currently initialized in >>> qemu_machine_init(). >>> >>> 2) Make qemu_machine_init() take no parameters and just reference >>> global state. >>> >> >> Any good idea how that global state is going to be stored? >> >> I just want to be able to launch a qemu on a different machine and >> migrate the "configuration" to it, for doing that, I really need what >> values are different from default or anything like that. So as you can >> see, I am very interested on that work. >> > > First step is to have everything go through QMP. If everything flows > through QMP, we have a gateway that we can focus on. > > What I'd like to do next is introduce the notion of a stateful config file. Any chance of reusing info qtree, QemuOpts, or other existing infrastructure for the config file? Config file set/get/format code tends to be just toggling variables and manipulating strings. There is similarity here with the option parsing and qdev properties. Stefan
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/23/2011 05:38 PM, Juan Quintela wrote:
Anthony Liguori wrote:
On 02/23/2011 05:00 PM, Juan Quintela wrote:
Anthony Liguori wrote:
The goal is to enable the monitor to run independently of whether the machine
has been created such that the monitor can be used to specify all of the
parameters for machine initialization.
Signed-off-by: Anthony Liguori
I agree that it is one step in the right direction, but we are still
calling qemu_machine_init() before calling the main_loop().
What is the plan from here?
1) Decouple QMP from qemu_machine_init(). This really requires the
introduction of the new QAPI server that exists outside of the chardev
infrastructure since chardevs are currently initialized in
qemu_machine_init().
2) Make qemu_machine_init() take no parameters and just reference
global state.
Any good idea how that global state is going to be stored?
I just want to be able to launch a qemu on a different machine and
migrate the "configuration" to it, for doing that, I really need what
values are different from default or anything like that. So as you can
see, I am very interested on that work.
First step is to have everything go through QMP. If everything flows
through QMP, we have a gateway that we can focus on.
What I'd like to do next is introduce the notion of a stateful config
file. This config file is essentially a global database that can be
used to store information about the guest from within QEMU.
So -name would become:
case QEMU_OPTION_name:
qmp_set_name(optarg, NULL);
break;
}
And qmp_set_name woudl be implemented as:
static const char *qemu_name;
void qmp_set_name(const char *name, Error **errp)
{
qemu_name = name;
}
char *qmp_query_name(Error **errp)
{
return qemu_strdup(qemu_name);
}
Now, to integrate this into the stateful config file, we would do
something like:
void qmp_set_name(const char *name, Error **errp)
{
qemu_set_config("global", "name", name);
}
char *qmp_query_name(Error **errp)
{
return qemu_strdup(qemu_get_config("global", "name"));
}
This is a simplistic example and more complex examples have not been
totally thought out, but this is where I'm looking to go.
Regards,
Anthony Liguori
Later, Juan.
Re: [Qemu-devel] Re: [PATCH] Split machine creation from the main loop
On 02/23/2011 05:00 PM, Juan Quintela wrote: Anthony Liguori wrote: The goal is to enable the monitor to run independently of whether the machine has been created such that the monitor can be used to specify all of the parameters for machine initialization. Signed-off-by: Anthony Liguori I agree that it is one step in the right direction, but we are still calling qemu_machine_init() before calling the main_loop(). What is the plan from here? 1) Decouple QMP from qemu_machine_init(). This really requires the introduction of the new QAPI server that exists outside of the chardev infrastructure since chardevs are currently initialized in qemu_machine_init(). 2) Make qemu_machine_init() take no parameters and just reference global state. 3) Teach all QMP functions to behave themselves if called before qemu_machine_init() 4) Introduce QMP function to call qemu_machine_init() 5) Introduce new command line flag to not automatically call qemu_machine_init() 6) Convert all command line options to just be QMP function calls (6) can be started right now. (1) comes with the QAPI merge. (2) is pretty easy to do after applying this patch. (3) is probably something that can be done shortly after (1). (4) and (5) really require everything but (6) to be in place before we can meaningful do it. I think we can lay out much of the ground work for this in 0.15 and I think we can have a total conversion realistically for 0.16. That means that by EOY, we could invoke QEMU with no options and do everything through QMP. Somewhere in all of this, we need to fit in a stateful (non-)config file too IMHO. Regards, Anthony Liguori Later, Juan.
