Re: [Qemu-devel] qdev for programmers writeup
On 07/11/2011 06:47 PM, Peter Maydell wrote: On 11 July 2011 16:29, Paolo Bonzinipbonz...@redhat.com wrote: On 07/11/2011 04:44 PM, Peter Maydell wrote: (Also if you have one bus type per board then you're still very limited in what you can do with -device because you can't plug in some random other sysbus device anyway.) I'm not talking about one bus type per board! I'm talking about as few as possible board-specific root devices, and sharing buses between boards as much as possible. Er, doesn't that just get you sysbus again? It does get you a bus that can be reused by devices. It doesn't get you a bus that is a pot-pourri of features, some of which are not even meaningful in the context of all boards (e.g. PIO), and some of which override the run-time reconfigurability mechanisms that qdev has built-in. By the way, while it's true that run-time reconfigurability does not buy you much in terms of adding devices---at least without a device tree in the guest---it can help in terms of removing devices for debugging. If a device only needs MMIO and no GPIO/IRQ pins, it probably can stay under SysBus. However, I don't believe the magic MMIO functionality of SysBus is useful, and I do think it should be replaced by properties. Also if you have a root device and it's not the CPU then something's a bit odd. (The CPU lives above the interrupt controller in the interrupt tree if you want to look at it like that.) If you consider the CPU to be hidden beyond sysbus, then yes, you do have CPU-SysBus-PIC. It is interesting that in the PC the devices below SysBus are indeed mostly managing interrupts: CPU-SysBus-LAPIC(s) IOAPIC HPET i440FX-pcihost fw_cfg I think the PC's fw_cfg device should move below the ISA bridge; and the HPET is there only because there is no single device for the northbridge chip. It should perhaps be more like CPU-SysBus-LAPIC(s) i440FX-nb-i440FX-pcihost IOAPIC HPET i8259 I think the real reason so many devices use sysbus is that it is basically I'm a device and I support some gpio signals and some memory mappings, which is just a very natural way to model a lot of things. I agree that sysbus is convenient sugar right now, and we need that sugar to be available at all levels (not just sysbus), but you don't need sysbus to express that. There is actually one thing that I'd save in sysbus, and that is IRQs. That is because GPIO pins provided by qdev work in one direction only. If you want to have interrupt/GPIO sources both towards the children and towards the parent, it doesn't work well. This is a nice niche that sysbus IRQs fit in; a GPIO chip can use gpio_in/gpio_out towards the children, and sysbus IRQs towards the parent, giving nice separation. And even if things tend to be tree-like, you still need to support arbitrary inter-wiring for the corner cases (like this MMC controller's 'card present' wire needs to connect to the board-register model's input). You can model trees with arbitrary interconnections, but not vice-versa. Yes, any slot/socket mechanism for run-time reconfigurability of GPIO or IRQ connections needs to take into account the possibility of connecting siblings (or even completely disconnected devices). Right now that is limited to C code. But since a GPIO/IRQ is simply a pointer, adding such a mechanism would be be just syntax to name the devices' GPIO/IRQ slots. But in any case you will need a preferred topology defined somewhere, because code needs more than a bunch of qemu_irqs. Since they know that the model is a tree, qdevified devices can exploit their parent-child relationship and you can use that to tie the parent and child in more specific ways with virtual functions. It's quite fundamental. This can stay even if you turn the preferred topology into a DAG, or into the superposition of many trees. (This view of the world, which I accept is not really qdev's, says that a bus is really just a conveniently named and manipulable bundle of connections.) I see qbuses as a conveniently named and pluggable set of callbacks (including qemu_irq callbacks whenever that's convenient). Alternatively, it's the point where the children's sockets are joined to the children's slots we're forced by qdev to make all sockets meet their slots in the same place---i.e. on the same qbus). Paolo
Re: [Qemu-devel] qdev for programmers writeup
On 11 July 2011 11:20, Paolo Bonzini pbonz...@redhat.com wrote: This is cool; more qdev documentation is really useful. One point I'd like clarification on is when you need to invent a new bus type. Sometimes it's pretty obvious because there's a real hardware bus type there (PCI, SCSI) that you're modelling. It's the edge cases I find confusing -- for instance, do we need a qbus for the connection between an SD card controller and the SD card model (hw/sd.c) ? There's a well defined pluggable interface between those two parts but there's only ever one SD card so a bus would be a bit odd. The first step is very important to achieve a quality conversion to qdev. QEMU includes partial conversions to qdev that have a large amount of SysBus devices, or devices that use DEFINE_PROP_PTR. In many cases, this is because the authors did not introduce a board-specific bus type to mediate access to the board resources. Together with such a bus type there should be a single root board-specific device that is attached to SysBus. An interrupt controller is usually a good candidate for this because it takes qemu_irqs from the outside, and can make good use of the specificities of SysBus. ...and this bit I don't understand. Why is SysBus a bad thing? It generally seems to me to be the right way to represent a bit of hardware which is fundamentally providing a memory mapped interface plus some GPIO lines. If you make your board use sysbus then it's easy to just plug in any existing sysbus device model qemu already has; if every board has its own bus type instead then this reuse just becomes unnecessarily harder, it seems to me. Also having the interrupt controller be the board specific device which you attach to sysbus seems totally wrong to me. This doesn't match hardware at all -- the interrupt controller deals with interrupt lines and isn't in the path for memory transactions at all. (The hierarchy for memory accesses may be completely different from how interrupts are wired, for that matter.) -- PMM
Re: [Qemu-devel] qdev for programmers writeup
On 07/11/2011 12:46 PM, Peter Maydell wrote:
One point I'd like clarification on is when you need to invent
a new bus type.
As rarely as possible, but as often as necessary? :P
New buses limit reusability of the models, but you need one whenever the
existing buses do not express how two devices interact.
Sometimes it's pretty obvious because there's
a real hardware bus type there (PCI, SCSI) that you're modelling.
It's the edge cases I find confusing -- for instance, do we need
a qbus for the connection between an SD card controller and the
SD card model (hw/sd.c) ? There's a well defined pluggable interface
between those two parts but there's only ever one SD card so a bus
would be a bit odd.
Perhaps not and you can use containment. A very similar case is the
16550 device, which has both an ISA (ioport-based) interface and a
generic memory-mapped interface. Anthony at some point argued that if
serial_mm_init was qdev-ified, one should describe SerialState as a
separate 16550 device, and then put a bus between {isa,mm}-serial and
thisg 16550 device. But perhaps including a SerialState struct into
both ISASerialState and the (hypothetical) MMSerialState is enough.
I think you have to look at the properties of the child device and the
interfaces between the devices. For SerialState, the properties would
be chardev and baudbase, and just a qemu_irq between the parent and
child. For SDState, the only property of the SDState would be the
blockdev, and again a couple GPIO pins between the two.
It probably would be feasible to separate the two. But then without a
clean plan about hiding such internal devices, it is likely a useless
complication for the user to see the existence of the SerialState and
SDState.
Together with such
a bus type there should be a single root board-specific device that is
attached to SysBus. An interrupt controller is usually a good candidate
for this because it takes qemu_irqs from the outside, and can make good
use of the specificities of SysBus.
...and this bit I don't understand. Why is SysBus a bad thing?
1) because SysBus devices are in general not accessible from the
command-line or configuration files.
2) because SysBus hardcodes in the source code some things that ought to
be device properties, for example the MMIO base address. The presence
of MMIO in sysbus_create_simple/sysbus_create_varargs is totally
unnecessary IMHO.
It generally seems to me to be the right way to represent a bit of
hardware which is fundamentally providing a memory mapped interface
plus some GPIO lines. If you make your board use sysbus then it's
easy to just plug in any existing sysbus device model qemu already
has; if every board has its own bus type instead then this reuse
just becomes unnecessarily harder, it seems to me.
That's true, but the only way to plug in those device models would be
with C code. You cannot just play with -device to reconfigure them.
It's not like SysBus has any problem; but it is right now the only
choice you have if you want a reusable model, and that means that
reusability can only be done at the cost of rebuilding QEMU.
For example, one reusable device is gpio_i2c. However, I cannot simply
take it and add it to a new board. I need to add glue like this:
/* dev is my GPIO device. */
i2c_dev = sysbus_create_simple(gpio_i2c, -1, NULL);
qdev_connect_gpio_out(i2c_dev, 0, qdev_get_gpio_in(dev, 29));
qdev_connect_gpio_out(dev, 3, qdev_get_gpio_in(i2c_dev, 0));
qdev_connect_gpio_out(dev, 4, qdev_get_gpio_in(i2c_dev, 1));
and recompile QEMU.
In fact, perhaps qdev_{connect_gpio_out,get_gpio_in} should never have
been public. Imagine we added to qdev GPIO properties and we used them
like this in gpio_i2c:
/* gpio_out=N means connect my 0th output pin to the parent's
N-th input pin. */
DEFINE_PROP_GPIO_OUT(gpio_out, 0),
/* gpio_in=N means connect my 0th input pin to the parent's
N-th output pin. */
DEFINE_PROP_GPIO_IN(gpio_in, 0),
DEFINE_PROP_GPIO_IN(gpio_clk, 1)
Then we define a GPIOBus that is really a bare-bones BusState, with no
MMIO and nothing. However, GPIO chips would expose one such bus, and a
lot of reusable components could be moved from SysBus to GPIOBus... and
get -device configuration at once! With this in place you can do:
-device gpio_i2c,gpio_out=29,gpio_in=3,gpio_clk=4
or in a configuration file:
[device gpio_i2c]
gpio_out = 29
gpio_in = 3
gpio_clk = 4
or if you really have to do it in C:
dev = qdev_create(gpiobus-bus, gpio_i2c);
qdev_set_prop_set_gpio_out(gpio_out, 29);
qdev_set_prop_set_gpio_in(gpio_in, 3);
qdev_set_prop_set_gpio_in(gpio_clk, 4);
qdev_init_nofail(dev);
Even the C code would already be an improvement, because the client code
has no idea of the pin numbers of gpio_i2c.
Note that gpio_i2c is already a well-defined device, and it uses only a
bunch of qdev
Re: [Qemu-devel] qdev for programmers writeup
On 11 July 2011 13:48, Paolo Bonzini pbonz...@redhat.com wrote:
On 07/11/2011 12:46 PM, Peter Maydell wrote:
Paolo wrote:
Together with such
a bus type there should be a single root board-specific device that is
attached to SysBus. An interrupt controller is usually a good candidate
for this because it takes qemu_irqs from the outside, and can make good
use of the specificities of SysBus.
...and this bit I don't understand. Why is SysBus a bad thing?
1) because SysBus devices are in general not accessible from the
command-line or configuration files.
2) because SysBus hardcodes in the source code some things that ought to be
device properties, for example the MMIO base address. The presence of MMIO
in sysbus_create_simple/sysbus_create_varargs is totally unnecessary IMHO.
MMIO base address shouldn't be a device property anyway -- it is
a property of the machine (board) model, ie how am I wiring up
these devices?, not a property of the device itself.
It generally seems to me to be the right way to represent a bit of
hardware which is fundamentally providing a memory mapped interface
plus some GPIO lines. If you make your board use sysbus then it's
easy to just plug in any existing sysbus device model qemu already
has; if every board has its own bus type instead then this reuse
just becomes unnecessarily harder, it seems to me.
That's true, but the only way to plug in those device models would be with C
code. You cannot just play with -device to reconfigure them.
I think the C source level reuse is more important and more useful
than plugging stuff around with -device, because in practice messing
about with the memory map of an embedded board isn't something people
typically want to do, because the kernel will have more-or-less
hardcoded where the devices are anyhow. (Also if you have one bus type
per board then you're still very limited in what you can do with -device
because you can't plug in some random other sysbus device anyway.) So if
we want to allow runtime configurability of boards like that we need
to do it by providing runtime configurability of sysbus devices.
(which approximately equates to runtime script-driven machine models).
In fact, perhaps qdev_{connect_gpio_out,get_gpio_in} should never have been
public. Imagine we added to qdev GPIO properties and we used them like this
in gpio_i2c:
/* gpio_out=N means connect my 0th output pin to the parent's
N-th input pin. */
DEFINE_PROP_GPIO_OUT(gpio_out, 0),
/* gpio_in=N means connect my 0th input pin to the parent's
N-th output pin. */
DEFINE_PROP_GPIO_IN(gpio_in, 0),
DEFINE_PROP_GPIO_IN(gpio_clk, 1)
I think it would be better to think of GPIO as a special (simple)
case of a generic desire to plug devices into each other (this is what
SystemC calls a port: basically an interface (API) defined so that
two things can make calls to arbitrary callback functions on the other
end of a connection). How ports are wired up is a property of the
machine model, not a property of the device at either end, and I think
it is better for the syntax not to mix up ports and properties.
Also, you want to be able to actually name your output pins, so
at a board level you can talk about wiring up sd_controller.wprot,
not sd_controller.gpio[3]... (Yes, this is to some extent just syntax
but I really don't want to end up with machine level modelling looking
like:
[device gpio_i2c]
gpio_out = 29
gpio_in = 3
gpio_clk = 4
...because that's just not really maintainable IMHO.)
Also having the interrupt controller be the board specific device
which you attach to sysbus seems totally wrong to me. This doesn't
match hardware at all -- the interrupt controller deals with
interrupt lines and isn't in the path for memory transactions at
all.
Well, it is clear that buses should be modelled after the way data flows.
But what is data? If data is what is being written, buses should be
modelled after the way memory transactions flow. If data is what is being
made available, buses are modelled more after the way interrupts flow.
GPIO is a strange thing in the middle. :)
Actually I think we should really be modelling buses only where the
hardware actually has a bus, ie a coherent collection of signals
between multiple devices such that you could in theory plug in
different devices in different slots. The memory transaction
related connections are much more buslike than interrupt wiring.
-- PMM
Re: [Qemu-devel] qdev for programmers writeup
On 07/11/2011 04:44 PM, Peter Maydell wrote: That's true, but the only way to plug in those device models would be with C code. You cannot just play with -device to reconfigure them. I think the C source level reuse is more important and more useful than plugging stuff around with -device, because in practice messing about with the memory map of an embedded board isn't something people typically want to do, because the kernel will have more-or-less hardcoded where the devices are anyhow. That's just because ARM doesn't (yet?) do device trees... :) (Also if you have one bus type per board then you're still very limited in what you can do with -device because you can't plug in some random other sysbus device anyway.) I'm not talking about one bus type per board! I'm talking about as few as possible board-specific root devices, and sharing buses between boards as much as possible. So if we want to allow runtime configurability of boards like that we need to do it by providing runtime configurability of sysbus devices. But we already have properties as a runtime configurability mechanism, and the additional functionality provided by SysBus is exactly to bypass it. In that sense SysBus is bad (if used widely as it is now). It is also bad because SysBus takes device metainformation (number of IRQ lines, size of MMIO and PIO areas) and puts it into per-device structure. I think it would be better to think of GPIO as a special (simple) case of a generic desire to plug devices into each other (this is what SystemC calls a port: basically an interface (API) defined so that two things can make calls to arbitrary callback functions on the other end of a connection). How ports are wired up is a property of the machine model, not a property of the device at either end But in practice you tend to have tree-like relationships. Even if it is just a two-level tree with a GPIO chip or interrupt controller at one end, and everything else at the other. Also, you want to be able to actually name your output pins, so at a board level you can talk about wiring up sd_controller.wprot, not sd_controller.gpio[3]... (Yes, this is to some extent just syntax but I really don't want to end up with machine level modelling looking like: It _is_ really just syntax. In my examples I used the raw values for GPIO pin numbers because that's what we have. So, right now there is no way to use symbolic values, but nothing forbids adding one. Or even automatically generating the QEMU .conf from something else. If you are using C code, of course you have #defines/enums. Well, it is clear that buses should be modelled after the way data flows. But what is data? If data is what is being written, buses should be modelled after the way memory transactions flow. If data is what is being made available, buses are modelled more after the way interrupts flow. GPIO is a strange thing in the middle. :) Actually I think we should really be modelling buses only where the hardware actually has a bus, ie a coherent collection of signals between multiple devices such that you could in theory plug in different devices in different slots. The memory transaction related connections are much more buslike than interrupt wiring. I'm not sure, perhaps the naming is bad. Abstracting from the name for a moment, unifying interrupts/GPIO with the qbus concept seemed natural to me in most of the cases I looked at. And it makes the handling of GPIO in qdev less schizophrenic; qdev seems to treat the qbus (whatever it is) as the One True Mechanism to join devices, and then adds GPIO on the side while providing no way to configure it. If you try to match the qdev tree with the tree of qemu_irq, the design becomes a lot more coherent. Paolo
Re: [Qemu-devel] qdev for programmers writeup
On 11 July 2011 16:29, Paolo Bonzini pbonz...@redhat.com wrote: On 07/11/2011 04:44 PM, Peter Maydell wrote: (Also if you have one bus type per board then you're still very limited in what you can do with -device because you can't plug in some random other sysbus device anyway.) I'm not talking about one bus type per board! I'm talking about as few as possible board-specific root devices, and sharing buses between boards as much as possible. Er, doesn't that just get you sysbus again? Also if you have a root device and it's not the CPU then something's a bit odd. (The CPU lives above the interrupt controller in the interrupt tree if you want to look at it like that.) So if we want to allow runtime configurability of boards like that we need to do it by providing runtime configurability of sysbus devices. But we already have properties as a runtime configurability mechanism, and the additional functionality provided by SysBus is exactly to bypass it. In that sense SysBus is bad (if used widely as it is now). It is also bad because SysBus takes device metainformation (number of IRQ lines, size of MMIO and PIO areas) and puts it into per-device structure. I think the real reason so many devices use sysbus is that it is basically I'm a device and I support some gpio signals and some memory mappings, which is just a very natural way to model a lot of things. It's also an easy to use API for machine models to instantiate and wire up both IRQs and memory mappings. If we want less use of sysbus then we need to have the basic qdev device model have much better support for doing this. I think it would be better to think of GPIO as a special (simple) case of a generic desire to plug devices into each other (this is what SystemC calls a port: basically an interface (API) defined so that two things can make calls to arbitrary callback functions on the other end of a connection). How ports are wired up is a property of the machine model, not a property of the device at either end But in practice you tend to have tree-like relationships. Even if it is just a two-level tree with a GPIO chip or interrupt controller at one end, and everything else at the other. Yes, but the tree-like relation of the memory transaction bus/interconnect isn't necessarily the same as the tree-like relation of the interrupt controllers; and the tree-like relation of the clocks might be different again. And even if things tend to be tree-like, you still need to support arbitrary inter-wiring for the corner cases (like this MMC controller's 'card present' wire needs to connect to the board-register model's input). You can model trees with arbitrary interconnections, but not vice-versa. Insisting that interrupts are always a tree also rules out ever having qemu support multiple distinct CPUs where some interrupt sources end up going to more than one CPU; I'd rather we didn't bake that kind of assumption into the core device model if possible. (This view of the world, which I accept is not really qdev's, says that a bus is really just a conveniently named and manipulable bundle of connections.) -- PMM
