I realized that I had forgot .num_irqchips = ARRAY_SIZE(config.irqchips) in the non-root cell config. After adding it to the non-root cell config and created the cell, I got the following:
irqchip_set_pending(), local_injection = 0, sender (current cpu) = 3 irqchip_set_pending()-> memory_barrier() irqchip_set_pending() -> send sgi to cpu:1, sgi_id:0, urrent cpu = 3 irqchip_handle_irq(cpu_data:0x0000ffffc021f000, irq_id:1), is_sgi(irq_id) = 1 irqchip_handle_irq() this_cpu_data() = 0x0000ffffc021f000, current cpu 1, cpu_data->cpu_id = 1 arch_handle_sgi SGI_EVENT(cpu_data=0x0000ffffc021f000, irqn=1), cpu id = 1, current cpu = 1 irqchip_handle_irq(cpu_data:0x0000ffffc0221000, irq_id:1), is_sgi(irq_id) = 1 irqchip_handle_irq() this_cpu_data() = 0x0000ffffc0221000, current cpu 2, cpu_data->cpu_id = 2 arch_handle_sgi SGI_EVENT(cpu_data=0x0000ffffc0221000, irqn=1), cpu id = 2, current cpu = 2 irqchip_handle_irq(cpu_data:0x0000ffffc0223000, irq_id:1), is_sgi(irq_id) = 1 irqchip_handle_irq() this_cpu_data() = 0x0000ffffc0223000, current cpu 3, cpu_data->cpu_id = 3 arch_handle_sgi SGI_EVENT(cpu_data=0x0000ffffc0223000, irqn=1), cpu id = 3, current cpu = 3 pci_cell_init - cell gic-demo-ivshmem, id = 1, num_pci_devices 1 pci_cell_init - pci type = 0 Adding PCI device 00:00.0 to cell "gic-demo-ivshmem" irqchip_handle_irq(cpu_data:0x0000ffffc0223000, irq_id:1), is_sgi(irq_id) = 1 irqchip_handle_irq() this_cpu_data() = 0x0000ffffc0223000, current cpu 3, cpu_data->cpu_id = 3 arch_handle_sgi SGI_EVENT(cpu_data=0x0000ffffc0223000, irqn=1), cpu id = 3, current cpu = 3 JAILHOUSE_CELL_CREATE: Cannot allocate memory If I change .vpci_irq_base = 144-32, I got I totally wrong behavior: irqchip_set_pending(), local_injection = 0, sender (current cpu) = 3 irqchip_set_pending()-> memory_barrier() irqchip_set_pending() -> send sgi to cpu:1, sgi_id:0, urrent cpu = 3 irqchip_handle_irq(cpu_data:0x0000ffffc0220000, irq_id:1), is_sgi(irq_id) = 1 irqchip_handle_irq() this_cpu_data() = 0x0000ffffc0220000, current cpu 1, cpu_data->cpu_id = 1 arch_handle_sgi SGI_EVENT(cpu_data=0x0000ffffc0220000, irqn=1), cpu id = 1, current cpu = 1 irqchip_handle_irq(cpu_data:0x0000ffffc0222000, irq_id:1), is_sgi(irq_id) = 1 irqchip_handle_irq() this_cpu_data() = 0x0000ffffc0222000, current cpu 2, cpu_data->cpu_id = 2 arch_handle_sgi SGI_EVENT(cpu_data=0x0000ffffc0222000, irqn=1), cpu id = 2, current cpu = 2 irqchip_handle_irq(cpu_data:0x0000ffffc0224000, irq_id:1), is_sgi(irq_id) = 1 irqchip_handle_irq() this_cpu_data() = 0x0000ffffc0224000, current cpu 3, cpu_data->cpu_id = 3 arch_handle_sgi SGI_EVENT(cpu_data=0x0000ffffc0224000, irqn=1), cpu id = 3, current cpu = 3 pci_cell_init - cell gic-demo-ivshmem, id = 1, num_pci_devices 1 pci_cell_init - pci type = 0, bdf = 0, shmem_region = 0 pci_cell_init - protocol = 0, cell name = gic-demo-ivshmem, peer_mem->size = 4096, phy add = 4278255616 Removing PCI device 00:00.0 from cell "ZynqMP-ZCU102" Adding PCI device 00:00.0 to cell "gic-demo-ivshmem" Created cell "gic-demo-ivshmem" pci_cell_init is not getting the correct values from the .pci_device, it is getting the values from the first memory region instead. Then, if I change .num_irqchips = 0, it gets the correct .pci_device values but then I dont have interrupts in the bare-metal cell. > Thanks Jan. > > So what I should do is to have a different IRQ for the root and non-root > cells? > > The root cell config has the following: > > ... > .vpci_irq_base = 136-32 > ... > .irqchips = { > /* GIC */ { > .address = 0xf9010000, > .pin_base = 32, > .pin_bitmap = { > 0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff, > }, > }, > }, > > I changed the non-root cell config to be similar to the linux-demo config: > > .vpci_irq_base = 140-32 > > and > > .irqchips = { > /* GIC */ { > .address = 0xf9010000, > .pin_base = 32, > .pin_bitmap = { > 1, 0, 0, 0, > }, > }, > }, > > > When I start the non-root cell, I can see the interrupt arriving in the root > cell, with irq_id = 136: > > ivshmem_register_mmio() mmio->address = 8, mmio->is_write = 0, current cpu = 3 > ivshmem_register_mmio() mmio->address = 0, mmio->is_write = 1, current cpu = 3 > arch_ivshmem_update_intx(ive=0x0000ffffc02450a0), > device->info->num_msix_vectors = 0, ctrl eg intx enable = 1 > ivshmem_register_mmio() mmio->address = 12, mmio->is_write = 1, current cpu = > 3 > @@@@@ ivshmem_remote_interrupt() ive->remote: 0x0000ffffc0245000, current cpu > = 3 > arch_ivshmem_trigger_interrupt(ive:0x0000ffffc0245000), irq_id:136, current > cpu = 3 > irqchip_set_pending(), local_injection = 0, sender (current cpu) = 3 > irqchip_set_pending() -> inside if(!cpu_data), irq_id = 136, gicd_base = > 0x0000000000140000, GICD_ISPENDR = 200, 16 > irqchip_handle_irq(cpu_data:0x0000ffffc021d000, irq_id:136), is_sgi(irq_id) = > 0 > irqchip_handle_irq() this_cpu_data() = 0x0000ffffc021d000, current cpu 0, > cpu_data->cpu_id = 0 > arch_handle_phys_irq calling irqchip_set_pending(cpu_data=0x0000ffffc021d000, > irqn=136), cpu_id = 0 > irqchip_set_pending(), local_injection = 1, sender (current cpu) = 0 > irqchip_set_pending() -> just return after irqchip.inject_irq, > cpu_data->cpu_id = 0 > irqchip_set_pending(), local_injection = 1, sender (current cpu) = 0 > irqchip_set_pending() -> just return after irqchip.inject_irq, > cpu_data->cpu_id = 0 > > When I write from Linux, the interrupt counter is not incremented > (/proc/interrupt remains the same) and I see that in Jailhouse generates an > interrupt with irq_id = 140 (the base for the non-root cell): > > Writing 1 to 0x7f910bf00c > ivshmem_register_mmio() mmio->address = 12, mmio->is_write = 1, current cpu = > 2 > @@@@@ ivshmem_remote_interrupt() ive->remote: 0x0000ffffc02450a0, current cpu > = 2 > arch_ivshmem_trigger_interrupt(ive:0x0000ffffc02450a0), irq_id:140, current > cpu = 2 > irqchip_set_pending(), local_injection = 0, sender (current cpu) = 2 > irqchip_set_pending() -> inside if(!cpu_data), irq_id = 140, gicd_base = > 0x0000000000140000, GICD_ISPENDR = 200, 16 > > > However, the interrupt is not handled by the non-root cell (cpu 3). I pass > 140 to gic_setup() and enable the irq by writing 0xffffffff to 0xfc100000: > > static void enable_irq(struct ivshmem_dev_data *d) > { > printk("IVSHMEM: Enabling IVSHMEM_IRQs registers = %p\n", d->registers); > mmio_write32(d->registers, 0xffffffff); > } > > It is not clear to me the relation between the root and non-root cells with > respect to the irqchips and vpci_irq_base configs. Clearly something is wrong > with my configs. > > Appreciate your help! > Giovani > > > > > Root cell and non-root cell are still sharing interrupts /wrt the > > virtual pci host controller and ivshmem. Just derive your inmate config > > from configs/arm64/zynqmp-zcu102-linux-demo[-2].c, those work. > > > > Jan > > -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.