Re: [PATCH 1/2] docs: update ivshmem device spec
Hello Claudio,
Sorry for the delay.
I am a bit short on time and will be offline for a week starting tonight.
I agree there are points that must be more clearly described (and I
agree that ivshmem code will most likely have to be cleaned up after this).
Restructuring the documentation with a optional section is a good idea
too.
I will work on this at my return.
Anyway, thanks for the review.
--
David Marchand
On 06/23/2014 04:18 PM, Claudio Fontana wrote:
Hi,
we were reading through this quickly today, and these are some of the questions
that
we think can came up when reading this. Answers to some of these questions we
think
we have figured out, but I think it's important to put this information into the
documentation.
I will quote the file in its entirety, and insert some questions inline.
Device Specification for Inter-VM shared memory device
--
The Inter-VM shared memory device is designed to share a region of memory to
userspace in multiple virtual guests.
What does to userspace mean in this context? The userspace of the host, or
the userspace in the guest?
What about The Inter-VM shared memory device is designed to share a memory region
(created on the host via the POSIX shared memory API) between multiple QEMU processes
running different guests. In order for all guests to be able to pick up the shared memory
area, it is modeled by QEMU as a PCI device exposing said memory to the guest as a PCI
BAR.
Whether in those guests the memory region is used in kernel space or userspace,
or there is even any meaning for those terms is guest-dependent I would think
(I think of an OSv here, where the application and kernel execute at the same
privilege level and in the same address space).
The memory region does not belong to any
guest, but is a POSIX memory object on the host.
Ok that's clear.
One thing I would ask is, but I don't know if it makes sense to mention here,
is who creates this memory object on the host?
I understand in some cases it's the contributed server (what you provide in contrib/), in some
cases it's the user of this device who has to write some server code for that, but
is it true that also the qemu process itself can create this memory object on its own, without
any external process needed? Is this the use case for host-guest only?
Optionally, the device may
support sending interrupts to other guests sharing the same memory region.
This opens a lot of questions here which are partly answered later (If I
understand correctly, not only interrupts are involved, but a complete
communication protocol involving registers in BAR0), but what about staying a
bit general here, like
Optionally, the device may also provide a communication mechanism between
guests sharing the same memory region. More details about that in the section
'OPTIONAL ivshmem guest to guest communication protocol'.
Thinking out loud, I wonder if this communication mechanism should be part of
this device in QEMU, or it should be provided at another layer..
The Inter-VM PCI device
---
*BARs*
The device supports three BARs. BAR0 is a 1 Kbyte MMIO region to support
registers. BAR1 is used for MSI-X when it is enabled in the device. BAR2 is
used to map the shared memory object from the host. The size of BAR2 is
specified when the guest is started and must be a power of 2 in size.
Are BAR0 and BAR1 optional? That's what I would think by reading the whole, but
I'm still not sure.
Am I forced to map BAR0 and BAR1 anyway? I don't think so, but..
If so, can we separate the explanation into the base shared memory feature, and
a separate section which explains the OPTIONAL communication mechanism, and the
OPTIONAL MSI-X BAR?
For example, say that I am a potential ivshmem user (which I am), and I am
interested in the shared memory but I want to use my own communication
mechanism and protocol between guests, can we make it so that I don't have to
wonder whether some of the info I read applies or not?
The solution to that I think is to put all the OPTIONAL parts into separate
sections.
*Registers*
Ok, so this should I think go into one such OPTIONAL sections.
The device currently supports 4 registers of 32-bits each. Registers
are used for synchronization between guests sharing the same memory object when
interrupts are supported (this requires using the shared memory server).
So use of BAR0 goes together with interrupts, and goes together with the shared
memory server (is it the one contributed in contrib/?)
The server assigns each VM an ID number and sends this ID number to the QEMU
process when the guest starts.
enum ivshmem_registers {
IntrMask = 0,
IntrStatus = 4,
IVPosition = 8,
Doorbell = 12
};
The first two registers are the interrupt mask and status registers. Mask and
status are only used with pin-based interrupts. They are unused with MSI
interrupts.
Re: [PATCH 1/2] docs: update ivshmem device spec
Hi,
we were reading through this quickly today, and these are some of the questions
that
we think can came up when reading this. Answers to some of these questions we
think
we have figured out, but I think it's important to put this information into the
documentation.
I will quote the file in its entirety, and insert some questions inline.
Device Specification for Inter-VM shared memory device
--
The Inter-VM shared memory device is designed to share a region of memory to
userspace in multiple virtual guests.
What does to userspace mean in this context? The userspace of the host, or
the userspace in the guest?
What about The Inter-VM shared memory device is designed to share a memory
region (created on the host via the POSIX shared memory API) between multiple
QEMU processes running different guests. In order for all guests to be able to
pick up the shared memory area, it is modeled by QEMU as a PCI device exposing
said memory to the guest as a PCI BAR.
Whether in those guests the memory region is used in kernel space or userspace,
or there is even any meaning for those terms is guest-dependent I would think
(I think of an OSv here, where the application and kernel execute at the same
privilege level and in the same address space).
The memory region does not belong to any
guest, but is a POSIX memory object on the host.
Ok that's clear.
One thing I would ask is, but I don't know if it makes sense to mention here,
is who creates this memory object on the host?
I understand in some cases it's the contributed server (what you provide in
contrib/), in some cases it's the user of this device who has to write some
server code for that, but is it true that also the qemu process itself can
create this memory object on its own, without any external process needed? Is
this the use case for host-guest only?
Optionally, the device may
support sending interrupts to other guests sharing the same memory region.
This opens a lot of questions here which are partly answered later (If I
understand correctly, not only interrupts are involved, but a complete
communication protocol involving registers in BAR0), but what about staying a
bit general here, like
Optionally, the device may also provide a communication mechanism between
guests sharing the same memory region. More details about that in the section
'OPTIONAL ivshmem guest to guest communication protocol'.
Thinking out loud, I wonder if this communication mechanism should be part of
this device in QEMU, or it should be provided at another layer..
The Inter-VM PCI device
---
*BARs*
The device supports three BARs. BAR0 is a 1 Kbyte MMIO region to support
registers. BAR1 is used for MSI-X when it is enabled in the device. BAR2 is
used to map the shared memory object from the host. The size of BAR2 is
specified when the guest is started and must be a power of 2 in size.
Are BAR0 and BAR1 optional? That's what I would think by reading the whole, but
I'm still not sure.
Am I forced to map BAR0 and BAR1 anyway? I don't think so, but..
If so, can we separate the explanation into the base shared memory feature, and
a separate section which explains the OPTIONAL communication mechanism, and the
OPTIONAL MSI-X BAR?
For example, say that I am a potential ivshmem user (which I am), and I am
interested in the shared memory but I want to use my own communication
mechanism and protocol between guests, can we make it so that I don't have to
wonder whether some of the info I read applies or not?
The solution to that I think is to put all the OPTIONAL parts into separate
sections.
*Registers*
Ok, so this should I think go into one such OPTIONAL sections.
The device currently supports 4 registers of 32-bits each. Registers
are used for synchronization between guests sharing the same memory object
when
interrupts are supported (this requires using the shared memory server).
So use of BAR0 goes together with interrupts, and goes together with the shared
memory server (is it the one contributed in contrib/?)
The server assigns each VM an ID number and sends this ID number to the QEMU
process when the guest starts.
enum ivshmem_registers {
IntrMask = 0,
IntrStatus = 4,
IVPosition = 8,
Doorbell = 12
};
The first two registers are the interrupt mask and status registers. Mask and
status are only used with pin-based interrupts. They are unused with MSI
interrupts.
Status Register: The status register is set to 1 when an interrupt occurs.
Mask Register: The mask register is bitwise ANDed with the interrupt status
and the result will raise an interrupt if it is non-zero. However, since 1 is
the only value the status will be set to, it is only the first bit of the mask
that has any effect. Therefore interrupts can be masked by setting the first
bit to 0 and unmasked by setting the first bit to
