Hi Markus,
On 4/23/24 6:39 AM, Markus Armbruster wrote:
Gustavo Romero <gustavo.rom...@linaro.org> writes:
Hi Markus,
Thanks for interesting in the ivshmem-flat device.
Bill Mills (cc:ed) is the best person to answer your question,
so please find his answer below.
On 2/28/24 3:29 AM, Markus Armbruster wrote:
Gustavo Romero <gustavo.rom...@linaro.org> writes:
[...]
This patchset introduces a new device, ivshmem-flat, which is similar to the
current ivshmem device but does not require a PCI bus. It implements the ivshmem
status and control registers as MMRs and the shared memory as a directly
accessible memory region in the VM memory layout. It's meant to be used on
machines like those with Cortex-M MCUs, which usually lack a PCI bus, e.g.,
lm3s6965evb and mps2-an385. Additionally, it has the benefit of requiring a tiny
'device driver,' which is helpful on some RTOSes, like Zephyr, that run on
memory-constrained resource targets.
The patchset includes a QTest for the ivshmem-flat device, however, it's also
possible to experiment with it in two ways:
(a) using two Cortex-M VMs running Zephyr; or
(b) using one aarch64 VM running Linux with the ivshmem PCI device and another
arm (Cortex-M) VM running Zephyr with the new ivshmem-flat device.
Please note that for running the ivshmem-flat QTests the following patch, which
is not committed to the tree yet, must be applied:
https://lists.nongnu.org/archive/html/qemu-devel/2023-11/msg03176.html
What problem are you trying to solve with ivshmem?
Shared memory is not a solution to any communication problem, it's
merely a building block for building such solutions: you invariably have
to layer some protocol on top. What do you intend to put on top of
ivshmem?
Actually ivshmem is shared memory and bi-direction notifications (in this case
a doorbell register and an irq).
Yes, ivshmem-doorbell supports interrupts. Doesn't change my argument.
This is the fundamental requirement for many types of communication but our
interest is for the OpenAMP project [1].
All the OpenAMP project's communication is based on shared memory and
bi-directional notification. Often this is on a AMP SOC with Cortex-As and
Cortex-Ms or Rs. However we are now expanding into PCIe based AMP. One example
of this is an x86 host computer and a PCIe card with an ARM SOC. Other
examples include two systems with PCIe root complex connected via a
non-transparent bridge.
The existing PCI based ivshmem lets us model these types of systems in a simple
generic way without worrying about the details of the RC/EP relationship or the
details of a specific non-transparent bridge. In fact the ivshmem looks to the
two (or more) systems like a non-transparent bridge with its own memory (and no
other memory access is allowed).
Right now we are testing this with RPMSG between two QEMU system where both
systems are cortex-a53 and both running Zephyr. [2]
We will expand this by switching one of the QEMU systems to either arm64 Linux
or x86 Linux.
So you want to simulate a heterogeneous machine by connecting multiple
qemu-system-FOO processes via ivshmem, correct?
An AMP SOC is one use case. A PCIe card with an embedded Cortex-M would
be another.
We (and others) are also working on a generic virtio transport that will work
between any two systems as long as they have shared memory and bi-directional
notifications.
On top of or adjacent to ivshmem?
On top of ivshmem. It is not the only use case but it is an important one.
I just gave a talk on this subject at EOSS. If you would like to look
at the slides they are here:
https://sched.co/1aBFm
Thanks,
Bill
Now for ivshmem-flat. We want to expand this model to include MCU like CPUs
and RTOS'es that don't have PCIe. We focus on Cortex-M because every open
source RTOS has an existing port for one of the Cortex-M machines already in
QEMU. However they don't normally pick the same one. If we added our own
custom machine for this, the QEMU project would push back and even if accepted
we would have to do a port for each RTOS. This would mean we would not test as
many RTOSes.
The ivshmem-flat is actually a good model for what a Cortex-M based PCIe card
would look like. The host system would see the connection as PCIe but to the
Cortex-M it would just appear as memory, MMR's for the doorbell, and an IRQ.
So even after we have a "roll your own machine definition from a file", I
expect ivshmem and ivshmem-flat to still be very useful.
[1] https://www.openampproject.org/
[2] Work in progress here:
https://github.com/OpenAMP/openamp-system-reference/tree/main/examples/zephyr/dual_qemu_ivshmem
--
Bill Mills
Principal Technical Consultant, Linaro
+1-240-643-0836
TZ: US Eastern
Work Schedule: Tues/Wed/Thur
