Updating:
I added
.num_msix_vectors = 1,
.iommu = 1,
to the root cell .pci_devices config, wrote a simple linux program to write to
the shared memory (0x800500000) and a simple inmate cell code to read from the
shared memory. It was able to read the values that were written.
The next step is how to generate interrupts among the cells. I tried to use the
pci functions (the same used in the ivshmem-demo) on the arm64 but got an
error. The lib/pci.c is not compiled by arm64.
How should I discover the pci device and send/receive interrupts on arm64?
Thanks
Giovani
> Thanks for the answers.
>
> So, I have changed the config. In the root cell I have the following now:
>
> /* IVSHMEM shared memory region for 00:00.0 */ {
> .phys_start = 0x800400000,
> .virt_start = 0x800400000,
> .size = 0x100000,
> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
> JAILHOUSE_MEM_ROOTSHARED, //added MEM_ROOTSHARED
> },
>
> .pci_devices = {
> /* 00:00.0 */ {
> .type = JAILHOUSE_PCI_TYPE_IVSHMEM,
> .bdf = 0 << 3,
> .bar_mask = {
> 0xffffff00, 0xffffffff, 0x00000000,
> 0x00000000, 0x00000000, 0x00000000,
> },
> .shmem_region = 3,
> .shmem_protocol = JAILHOUSE_SHMEM_PROTO_UNDEFINED,
> //changed from JAILHOUSE_SHMEM_PROTO_VETH to JAILHOUSE_SHMEM_PROTO_UNDEFINED
> },
> },
>
> In the inmate cell, I have the following:
>
> .mem_regions = {
> ......
> .....
> /* IVSHMEM shared memory region for 00:00.0 */ {
> .phys_start = 0x800400000,
> .virt_start = 0x800400000,
> .size = 0x100000,
> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
> JAILHOUSE_MEM_ROOTSHARED, //added JAILHOUSE_MEM_ROOTSHARED
> },
> },
>
> Considering just the configuration of the cells, are they correct?
>
> So correct me if I am wrong. I will need to write a driver in the root-cell
> (Linux) to send and receive interrupts to/from the shared ivshmem region. The
> mapped vpci region contains only the registers of the device:
>
> 00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
> Subsystem: Red Hat, Inc Inter-VM shared memory
> Flags: fast devsel, IRQ 38
> Memory at fc100000 (64-bit, non-prefetchable) [size=256]
>
> In this case, the address 0xfc100000.
>
> Is the address (0x800400000) that I configured in the cell config file
> actually the payload memory region?
>
> Also, in the another inmate, how can I access the mapped region? I noticed
> that there is an ivshmem-demo for x86, but there isn't for arm64.
>
> Thanks
>
> > Hi,
> >
> > what you see there are the registers of the ivshmem device, not the
> > payload memory region. You will have to map the paddr you have in your
> > cell-config. That paddr can be found in the config space as well, is
> > just not a regular PCI BAR.
> >
> > I would suggest to write a uio driver and base it on
> > https://github.com/henning-schild-work/ivshmem-guest-code/tree/jailhouse
> >
> > Here is the bit that finds the region
> > https://github.com/henning-schild-work/ivshmem-guest-code/blob/jailhouse/kernel_module/uio/uio_ivshmem.c#L96
> >
> > Also see:
> > https://github.com/henning-schild-work/ivshmem-guest-code/blob/jailhouse/README.jailhouse
> >
> > And you should not use JAILHOUSE_SHMEM_PROTO_VETH because you are not
> > running a network-device on top of your shmem. Use _UNDEFINED or
> > _CUSTOM instead.
> >
> > Henning
> >
> > Am Tue, 3 Apr 2018 10:47:31 -0700
> > schrieb Giovani Gracioli <>:
> >
> > > Just another info, after enabling the root cell, I can see the
> > > virtual pci devices with lspci -v:
> > >
> > > 00:00.0 Unassigned class [ff01]: Red Hat, Inc Inter-VM shared memory
> > > Subsystem: Red Hat, Inc Inter-VM shared memory
> > > Flags: fast devsel
> > > Memory at fc100000 (64-bit, non-prefetchable) [disabled]
> > > [size=256] Memory at <unassigned> (64-bit, non-prefetchable)
> > > [disabled] Capabilities: [50] MSI-X: Enable- Count=1 Masked-
> > >
> > > 00:01.0 Unassigned class [ff01]: Red Hat, Inc Inter-VM shared memory
> > > Subsystem: Red Hat, Inc Inter-VM shared memory
> > > Flags: fast devsel
> > > Memory at fc100100 (64-bit, non-prefetchable) [disabled]
> > > [size=256]
> > >
> > >
> > >
> > > > Hello,
> > > >
> > > > I would like to share buffers among cells on arm64 (Xilinx
> > > > ultrascale+). The documentation suggests the use of the ivshmem.
> > > >
> > > > In order to use ivshmem, I changed the root cell config as follows
> > > > (it is based on the zynqmp-zcu102.c original file):
> > > >
> > > > - In the .mem_regions:
> > > >
> > > > /* IVSHMEM shared memory region for 00:00.0 */ {
> > > > .phys_start = 0x800400000,
> > > > .virt_start = 0x800400000,
> > > > .size = 0x100000,
> > > > .flags = JAILHOUSE_MEM_READ |
> > > > JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED, //added
> > > > JAILHOUSE_MEM_ROOTSHARED },
> > > >
> > > > - In the .pci_devices:
> > > >
> > > > /* 00:00.0 */ {
> > > > .type = JAILHOUSE_PCI_TYPE_IVSHMEM,
> > > > .bdf = 0 << 3,
> > > > .bar_mask = {
> > > > 0xffffff00, 0xffffffff, 0x00000000,
> > > > 0x00000000, 0x00000000, 0x00000000,
> > > > },
> > > > .shmem_region = 3,
> > > > .shmem_protocol =
> > > > JAILHOUSE_SHMEM_PROTO_VETH, .num_msix_vectors = 1,
> > > > .iommu = 1,
> > > > },
> > > >
> > > > Then, I load the cell:
> > > >
> > > > jailhouse enable zynqmp-zcu102-ivshmem.cell
> > > >
> > > > Initializing Jailhouse hypervisor v0.8 (37-g1fa9001) on CPU 1
> > > > Code location: 0x0000ffffc0200060
> > > > Page pool usage after early setup: mem 33/995, remap 64/131072
> > > > Initializing processors:
> > > > CPU 1... OK
> > > > CPU 2... OK
> > > > CPU 0... OK
> > > > CPU 3... OK
> > > > Adding virtual PCI device 00:00.0 to cell "ZynqMP-ZCU102"
> > > > Adding virtual PCI device 00:01.0 to cell "ZynqMP-ZCU102"
> > > > Page pool usage after late setup: mem 42/995, remap 69/131072
> > > > Activating hypervisor
> > > >
> > > > Then, I wrote a simple user-space program that maps the virtual PCI
> > > > ivshmem region to user using mmap:
> > > >
> > > > memfd = open("/dev/mem", O_RDWR | O_SYNC);
> > > >
> > > > mapped_base = mmap(0, MEM_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED,
> > > > memfd, 0xfc100000 & ~MEM_MASK);
> > > >
> > > > mapped_dev_base = mapped_base + (dev_base & MEM_MASK);
> > > >
> > > > for(int i = 0; i < 8; i += 4) {
> > > > *((volatile unsigned int *) (mapped_dev_base + i)) = 1;
> > > > printf("Address: %p, Read valeu = %d\n", (void
> > > > *)(mapped_dev_base + i), *((unsigned int *) (mapped_dev_base +
> > > > i))); }
> > > >
> > > > When I ran this program in the root cell, I got the following
> > > > Unhandled trap:
> > > >
> > > > ./ivshmem_test
> > > > /dev/mem opened.
> > > > Memory mapped at address 0x7f8dd87000.
> > > > Unhandled data write at 0xfc100000(4)
> > > >
> > > > FATAL: unhandled trap (exception class 0x24)
> > > > Cell state before exception:
> > > > pc: 0000000000400904 lr: 00000000004008d8 spsr: 80000000 EL0
> > > > sp: 0000007fceff8df0 esr: 24 1 1810046
> > > > x0: 0000007f8dd87000 x1: 0000000000000001 x2: 0000000000000001
> > > > x3: 0000000000000000 x4: 0000000040100401 x5: 5404000000000000
> > > > x6: 000000001aec1037 x7: 0000000000000000 x8: 0000000000000040
> > > > x9: ffffff80ffffffc8 x10: 0000007fceff8df0 x11: 0000007fceff8df0
> > > > x12: 00000000000003f3 x13: 0000000000000000 x14: 0000000000000000
> > > > x15: 0000007f8dd8ecc0 x16: 0000000000000000 x17: 0000007f8dc53240
> > > > x18: 0000000000000a03 x19: 00000000004009a8 x20: 0000000000000000
> > > > x21: 0000000000000000 x22: 0000000000000000 x23: 0000000000000000
> > > > x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000
> > > > x27: 0000000000000000 x28: 0000000000000000 x29: 0000007fceff8df0
> > > >
> > > > Parking CPU 2 (Cell: "ZynqMP-ZCU102")
> > > >
> > > > Does that make sense? Am I missing something in the root cell
> > > > config or the user-space program is wrong?
> > > >
> > > > Best regards
> > >
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
