Re: RTOS inmate misses interrupts
Hi Ralf,
Do you know of a difference in the way SPI and PPI are treated by the
hypervisor?
A timer that is specific for a certain core is wired to GIC by PPI. Would
its processing be influenced by heavy load running on another core?
The GIC is facing the same load, but handles PPI instead of SPI. What would
be the implications on performance?
Thanks a lot,
Nir.
On Wed, Dec 14, 2022 at 11:09 AM Rasty Slutsker wrote:
> Hi,
> Exactly I reference to 'jailhouse cell stats'
> We enabled TSC in Hypervisor init, and printed value to dmsg.
> But *after* start of Linux it gets disabled again.
> *** Is there a place in hypervisor that runs at kernel space after start
> of linux where we can put re-enabling of TSC?
>
> We would like to measure duration of Hypervisor ISR with TSC and look for
> anomalies.
> Anomalies like unusual duration of ISR processing and Irregularities in
> ISR entry for particular request number.
>
> Other suggestions would be highly appreciated.
>
> Thanks
> Rasty
>
> On Tuesday, December 13, 2022 at 11:27:50 PM UTC+2 Ralf Ramsauer wrote:
>
>> Hi Rasty,
>>
>> (reply-to-all :) )
>>
>> On 13/12/2022 19:31, Rasty Slutsker wrote:
>> > We learned how you export some statistics from Jailhouse as you guys do
>> > and added 3 variables
>>
>> You reference to 'jailhouse cell stats'?
>>
>> > 1. At the entry Jailhouse IRQ (if irq==xxx counter ++)
>> > 2. At injection point of the same IRQ to inmate, still in Jailhouse
>> > 3. At the beginning of ISR in inmate (RTOS).
>>
>> Ok, but that just counts the interrupts, and not the occuring delays,
>> right?
>>
>> >
>> > We let system run, introduce some load to linux. We see physical
>> effects
>> > that suggest that we lose interrupts.
>> > We confirm it with difference in performance counters (gaps in quanta
>> of
>> > 30 uSecs) that we sample in inmate ISR.
>> > Than we kill interrupt.
>> > All 3 counters are the same. Amount matches interrupt rate.
>> > My conclusion that Interrupt request is lost.
>>
>> Could be the case. I don't know what happens if the jitter gets too long
>> between interrupts.
>>
>> >
>> > Another question.
>> > We try to read CPU time stamp counter from Jailhouse ISR . We get 0.
>> > mrc p15, #0, r0, c9, c13, #0
>>
>> That's the PMCNTR, right?
>>
>> >
>> > Any idea why? Do you have some code for that?
>>
>> Uhm, I would have to read the reference manual as well. Does reading the
>> TSC work in Linux? Maybe it has to be activated or enabled for the
>> hypervisor?
>>
>>
>> https://developer.arm.com/documentation/ddi0406/b/Debug-Architecture/Debug-Registers-Reference/Performance-monitor-registers/c9--Count-Enable-Set-Register--PMCNTENSET-?lang=en
>>
>> Thanks,
>> Ralf
>>
>> >
>> > Best regards
>> > Rasty
>> >
>> >
>> >
>> > On Tuesday, December 13, 2022 at 3:47:25 PM UTC+2 Ralf Ramsauer wrote:
>> >
>> > Hi Rasty,
>> >
>> > Please reply-to-all, then your reply will also pop up in my Inbox.
>> >
>> > On 10/12/2022 08:52, Rasty Slutsker wrote:
>> > > Hi,
>> > > We did some performance measurements.
>> > > Added counters in 3 places (per Irq source)
>> > > 1. entry to jailhouse ISR
>> > > 2. dispatch of interrupt to particular vector to particular core
>> > > 3. in RTOS isr.
>> >
>> > Okay. How do you read and dump them? I hope after everything is done.
>> >
>> > Take care that if you dump them immediately to the uart, this will
>> > consume a lot of time and cause significant delay. ('heavy' printk
>> > logic
>> > + busy waiting for the uart)
>> >
>> > >
>> > > *We see that all 3 counters have the same value*, but we measure
>> > time
>> >
>> > huh? What counters do you access? There's something odd if they hold
>> > the
>> > same value at different places.
>> >
>> > > gaps in RTOS in ISR invocation times, sometimes upto 60 uSec.
>> > >
>> > > It means that either
>> > > a) interrupt request is lost. But, according to setup it is
>> > > edge-triggered, it cannot be lost, just delayed.
>> > > b) there is a delay of more than 60 usec in jailhouse ISR.
>> > >
>> > > questions:
>> > > 1. Is it possible that jailhouse interrupt dispatching routine
>> > enters
>> > > some loop that takes considerable amount of time?
>> >
>> > If you use printk during dispatching for debugging - yes. Otherwise, I
>> > guess no. Not 60µs.
>> >
>> > > 2. What would be explanation of interrupt latency of 60 Secs?
>> > Even if we
>> > > take into account cache line refill we get much lower number,
>> > which do
>> > > not reach tens uSecs.
>> >
>> > Ack, I would definitely not expect 60µs delay for IRQ reinjection.
>> >
>> > Thanks
>> > Ralf
>> >
>> > >
>> > > Best regards
>> > > Rasty
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Tuesday, December 6, 2022 at 6:01:20 PM UTC+2 Ralf Ramsauer
>> > wrote:
>> > >
>> > > Hi,
>> > >
>> > > On 05/12/2022 17:30, Rasty Slutsker wrote:
>> > > > Hi Ralf,
>> > > > Thank you for the answer.
>> > > > We have periodic interrupt each 30 u(!)Sec. Linux cannot deal
>> > > with such
>> > > > rate, so we need hypervisor/RTOS.
Re: Interrupt Latency in RTOS inmate cell
I have a simple memory communication region between the 2 cells, not IVSHMEM. I use a kernel module that emulates ethernet over this memory region, but the jitter still happens even when I don't load this kernel module. The jitter still exists when completely disabling UART1 (Linux debug console) in the dts. The storage device is a eMMC. Maybe RCU or swapping are related? Thanks, Nir. On Tuesday, September 3, 2019 at 3:19:10 PM UTC+3, Henning Schild wrote: > > Am Tue, 3 Sep 2019 02:28:13 -0700 > schrieb Nir Geller >: > > > Hi Henning > > > > When I execute "sync" I get a spike on jitter. > > When executing > > echo 1 > /proc/sys/vm/drop_caches > > or > > echo 2 > /proc/sys/vm/drop_caches > > or > > echo 3 > /proc/sys/vm/drop_caches > > No spike occurs. > > Not those caches. The CPUs caches. If your two cores share i.e. a > layer3 cache one core can evict all warm entries from the other. > Forcing that other to go back to slow RAM and get it again. In the > worst case you share and NUMA kicks in, which gets you twice the > slowness of RAM. > > Do you share any memory i.e. a ivshmem region? > > > I'm not familiar with AMBA, so I'm trying to learn the subject now. I > > guess you mean giving higher priority to core1 > > in attempt to narrow DMA latency? > > Yes i thought about DMA latency. But if that also happens for sysfs or > proc, there is no IO and no DMA. > What is your storage backing anyways? NFS, some filesystem on some mass > storage ? > > Maybe your Linux UART has some shared resources with with the GPIO? Can > you silence the UART and operate that Linux with i.e. ssh? > > Henning > > > Thanks, > > > > Nir. > > > > On Monday, September 2, 2019 at 7:57:16 PM UTC+3, Henning Schild > > wrote: > > > > > > A good way of estimating the cost of someone messing with ones > > > caches is to flush them yourself in your test and see how much that > > > costs you. > > > > > > I am not sure how complicated it is to read performance counters on > > > ARM, but those might give a clue. > > > > > > AMBA QoS can maybe help raise the IO prio of your RT-cell. If a DMA > > > request is what is slowing down your GPIO. > > > I have never used that but maybe you can configure it from Linux > > > before you enable Jailhouse, and make sure that your root cell > > > config blocks future access to the configuration mechanism. > > > > > > Henning > > > > > > > > > > > > Am Mon, 2 Sep 2019 17:27:47 +0200 > > > schrieb Ralf Ramsauer >: > > > > > > > Hi, > > > > > > > > On 9/2/19 5:11 PM, Nir Geller wrote: > > > > > CPUFREQ is set to performance, and I'm setting scaling_min_freq > > > > > to the highest available frequency (150) > > > > > CPU idling is disabled > > > > > > > > > > Now I see that executing a simple "cat somefile" on the command > > > > > line causes a spike in jitter > > > > > > > > only for the first time or also for consecutive calls on the same > > > > file? IOW, can you observe/trigger the spike when somefile is in > > > > your page cache? > > > > > > > > Does the non-root cell share any devices with the root cell? > > > > (e.g. debug UART) > > > > > > > > Ralf > > > > > > > > > > > > > > On Monday, September 2, 2019 at 5:51:24 PM UTC+3, Henning > > > > > Schild wrote: > > > > > > > > > > Am Mon, 2 Sep 2019 06:12:00 -0700 > > > > > schrieb Nir Geller >: > > > > > > > > > > > I created a kernel module that catches/releases a > > > > > > spinlock and disables/enables preemption, and it had no > > > > > > observable effect on the jitter, however, > > > > > > the operations insmod and rmmod definitely cause spikes > > > > > > in jitter. > > > > > > > > > > > > Any pointers? > > > > > > > > > > Do you have any power management features enabled in that > > > > > Linux? > > > > > > > > > > Henning > > > > > > > > > > > Thanks. > > >
Re: Interrupt Latency in RTOS inmate cell
Hi Henning When I execute "sync" I get a spike on jitter. When executing echo 1 > /proc/sys/vm/drop_caches or echo 2 > /proc/sys/vm/drop_caches or echo 3 > /proc/sys/vm/drop_caches No spike occurs. I'm not familiar with AMBA, so I'm trying to learn the subject now. I guess you mean giving higher priority to core1 in attempt to narrow DMA latency? Thanks, Nir. On Monday, September 2, 2019 at 7:57:16 PM UTC+3, Henning Schild wrote: > > A good way of estimating the cost of someone messing with ones caches > is to flush them yourself in your test and see how much that costs you. > > I am not sure how complicated it is to read performance counters on > ARM, but those might give a clue. > > AMBA QoS can maybe help raise the IO prio of your RT-cell. If a DMA > request is what is slowing down your GPIO. > I have never used that but maybe you can configure it from Linux before > you enable Jailhouse, and make sure that your root cell config blocks > future access to the configuration mechanism. > > Henning > > > > Am Mon, 2 Sep 2019 17:27:47 +0200 > schrieb Ralf Ramsauer >: > > > Hi, > > > > On 9/2/19 5:11 PM, Nir Geller wrote: > > > CPUFREQ is set to performance, and I'm setting scaling_min_freq to > > > the highest available frequency (150) > > > CPU idling is disabled > > > > > > Now I see that executing a simple "cat somefile" on the command line > > > causes a spike in jitter > > > > only for the first time or also for consecutive calls on the same > > file? IOW, can you observe/trigger the spike when somefile is in your > > page cache? > > > > Does the non-root cell share any devices with the root cell? (e.g. > > debug UART) > > > > Ralf > > > > > > > > On Monday, September 2, 2019 at 5:51:24 PM UTC+3, Henning Schild > > > wrote: > > > > > > Am Mon, 2 Sep 2019 06:12:00 -0700 > > > schrieb Nir Geller >: > > > > > > > I created a kernel module that catches/releases a spinlock and > > > > disables/enables preemption, and it had no observable effect > > > > on the jitter, however, > > > > the operations insmod and rmmod definitely cause spikes in > > > > jitter. > > > > > > > > Any pointers? > > > > > > Do you have any power management features enabled in that Linux? > > > > > > Henning > > > > > > > Thanks. > > > > > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "Jailhouse" group. > > > To unsubscribe from this group and stop receiving emails from it, > > > send an email to jailho...@googlegroups.com > > > <mailto:jailhouse-dev+unsubscr...@googlegroups.com >. > > > To view this discussion on the web visit > > > > https://groups.google.com/d/msgid/jailhouse-dev/a3c23a6a-95ee-4baa-9714-229c84d9d5b7%40googlegroups.com > > > > < > https://groups.google.com/d/msgid/jailhouse-dev/a3c23a6a-95ee-4baa-9714-229c84d9d5b7%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > > > > -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/7ab0b502-2efe-41fe-ae9e-ec2c8f2c1b7a%40googlegroups.com.
Re: Interrupt Latency in RTOS inmate cell
Hi Ralf, I get a spike on jitter when executing consecutive "cat somefile". The spike happens when opening a file on the storage device, when opening a file on a tmpfs, when opening a file on procfs and even when executing "cat" on a file that doesn't exist at all. The root and inmate cells do not share resources and the debug consoles are directed to different UARTs. u-boot, Linux serial console and the hypervisor debug console are on UART1, and the inmate is on UART10. Thanks, Nir. On Monday, September 2, 2019 at 6:27:50 PM UTC+3, Ralf Ramsauer wrote: > > Hi, > > On 9/2/19 5:11 PM, Nir Geller wrote: > > CPUFREQ is set to performance, and I'm setting scaling_min_freq to the > > highest available frequency (150) > > CPU idling is disabled > > > > Now I see that executing a simple "cat somefile" on the command line > > causes a spike in jitter > > only for the first time or also for consecutive calls on the same file? > IOW, can you observe/trigger the spike when somefile is in your page > cache? > > Does the non-root cell share any devices with the root cell? (e.g. debug > UART) > > Ralf > > > > > On Monday, September 2, 2019 at 5:51:24 PM UTC+3, Henning Schild wrote: > > > > Am Mon, 2 Sep 2019 06:12:00 -0700 > > schrieb Nir Geller >: > > > > > I created a kernel module that catches/releases a spinlock and > > > disables/enables preemption, and it had no observable effect on > the > > > jitter, however, > > > the operations insmod and rmmod definitely cause spikes in jitter. > > > > > > Any pointers? > > > > Do you have any power management features enabled in that Linux? > > > > Henning > > > > > Thanks. > > > > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Jailhouse" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to jailho...@googlegroups.com > > <mailto:jailhouse-dev+unsubscr...@googlegroups.com >. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/jailhouse-dev/a3c23a6a-95ee-4baa-9714-229c84d9d5b7%40googlegroups.com > > > < > https://groups.google.com/d/msgid/jailhouse-dev/a3c23a6a-95ee-4baa-9714-229c84d9d5b7%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/69047717-186b-4e56-b8af-50294e493caf%40googlegroups.com.
Re: Interrupt Latency in RTOS inmate cell
CPUFREQ is set to performance, and I'm setting scaling_min_freq to the highest available frequency (150) CPU idling is disabled Now I see that executing a simple "cat somefile" on the command line causes a spike in jitter On Monday, September 2, 2019 at 5:51:24 PM UTC+3, Henning Schild wrote: > > Am Mon, 2 Sep 2019 06:12:00 -0700 > schrieb Nir Geller >: > > > I created a kernel module that catches/releases a spinlock and > > disables/enables preemption, and it had no observable effect on the > > jitter, however, > > the operations insmod and rmmod definitely cause spikes in jitter. > > > > Any pointers? > > Do you have any power management features enabled in that Linux? > > Henning > > > Thanks. > > > > -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/a3c23a6a-95ee-4baa-9714-229c84d9d5b7%40googlegroups.com.
Re: Interrupt Latency in RTOS inmate cell
I created a kernel module that catches/releases a spinlock and disables/enables preemption, and it had no observable effect on the jitter, however, the operations insmod and rmmod definitely cause spikes in jitter. Any pointers? Thanks. -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/bce2e24b-2063-4ce1-a1c9-c99efcd2ba5d%40googlegroups.com.
Interrupt Latency in RTOS inmate cell
Hi there, I'm using Jailhouse 0.7 on a AM5728 based custom board, on which I'm running Linux on A15 core0 and TI-RTOS on A15 core1. Linux kernel is 4.14.79 with RT-PREEMPT patch, supplied by TI. On the TI-RTOS side I have an interrupt driven by a togglled GPIO at duty cycle of 31.25 usec, and I'm experiencing jitter of up to 40 usec. This happens when both Linux and RTOS are not under any load. When running the TI-RTOS natively on A15 core0 the jitter is stable and doesn't get higher than 12 usec, which is acceptable. Also, I started the hypervisor and the TI-RTOS app, then I made the hypervisor park core0 by attempting to read a memory address that Linux is not allowed to access and again on core1 the jitter is stable and doesn't get higher than 12 usec. I guess that Linux is causing the interference. Does Linux postpone/disable interrupts on core 1? Can it be avoided? What else might cause the jitter? Does it make sense to port Jailhouse to a newer version? Thanks, Nir. -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/361d6548-e251-4e1e-a03b-cdcbfda1b67d%40googlegroups.com.
Re: ARMv7: using uio_ivshmem to send interrupts between cells
Hi, I figure there's missing code related to uio_ivshmem in Jailhouse v0.7. I migrated to the latest jailhouse version, branch master. I'm using RT_PREEMPT kernel 4.14.79 When I enable the root cell it seems like the hypervisor is not initialized. The only prints I get: [ 35.272718] OF: PCI: host bridge /pci@0 ranges: [ 35.306648] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010 [ 35.319993] pci-host-generic 3000.pci: ECAM at [mem 0x3000-0x300f] for [bus 00] [ 35.330762] pci-host-generic 3000.pci: PCI host bridge to bus 0001:00 [ 35.337600] pci_bus 0001:00: root bus resource [bus 00] [ 35.347049] pci_bus 0001:00: root bus resource [mem 0x3010-0x30101fff] [ 35.355943] pci 0001:00:00.0: [1af4:1110] type 00 class 0xff [ 35.355981] pci 0001:00:00.0: reg 0x10: [mem 0x-0x00ff 64bit] [ 35.356355] PCI: bus0: Fast back to back transfers disabled [ 35.367812] pci 0001:00:00.0: BAR 0: assigned [mem 0x3010-0x301000ff 64bit] [ 35.376191] virtio-pci 0001:00:00.0: enabling device ( -> 0002) [ 35.382777] uio_ivshmem 0001:00:00.0: using jailhouse mode [ 35.389246] The Jailhouse is opening. [ 36.001021] Created Jailhouse cell "AM5728-IDK-RTOS" When enabling jailhouse v0.7 I get prints that prove entry() was indeed invoked and the hypervisor is initialized: Initializing Jailhouse hypervisor v0.7 (222-g5e1b3ea-dirty) on CPU 0 entry(): started Code location: 0xf040 Page pool usage after early setup: mem 30/4072, remap 32/131072 Initializing processors: CPU 0... OK CPU 1... OK Adding virtual PCI device 00:00.0 to cell "AM5728-IDK-LINUX" Page pool usage after late setup: mem 44/4072, remap 38/131072 Activating hypervisor [ 54.734879] OF: PCI: host bridge /vpci@0 ranges: [ 54.768094] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010 [ 54.774264] pci-host-generic 3000.vpci: ECAM at [mem 0x3000-0x300f] for [bus 00] [ 54.783014] pci-host-generic 3000.vpci: PCI host bridge to bus 0001:00 [ 54.790008] pci_bus 0001:00: root bus resource [bus 00] [ 54.795264] pci_bus 0001:00: root bus resource [mem 0x3010-0x30101fff] [ 54.802663] PCI: bus0: Fast back to back transfers disabled [ 54.808283] pci 0001:00:00.0: BAR 0: assigned [mem 0x3010-0x301000ff 64bit] [ 54.815988] virtio-pci 0001:00:00.0: enabling device ( -> 0002) [ 54.829614] uio_ivshmem 0001:00:00.0: using jailhouse mode [ 54.853170] The Jailhouse is opening. Can I apply some fix to make the hypervisor initialize? Thanks, Nir. -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ARMv7: using uio_ivshmem to send interrupts between cells
On Tuesday, April 9, 2019 at 8:06:58 PM UTC+3, João Reis wrote: > domingo, 7 de Abril de 2019 às 13:00:48 UTC+1, Nir Geller escreveu: > > On Monday, April 1, 2019 at 2:18:07 PM UTC+3, Nir Geller wrote: > > > Hi there, > > > > > > > > > > > > > > > I'm developing over ARMv7 Sitara IDK am5728 > > > > > > > > > I'm using the jailhouse hypervisor v0.8 and I'm able to run Linux in a > > > root cell on core 0 and a TI RTOS as an inmate on core 1. > > > > > > > > > # jailhouse --version > > > Jailhouse management tool v0.8 > > > > > > > > > A shared memory region seems to be working fine. > > > I've set up uio_ivshmem and now I'm trying to send interrupts between > > > cells. > > > > > > > > > [ 4600.643916] OF: PCI: host bridge /vpci@0 ranges: > > > [ 4600.648578] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010 > > > [ 4600.654583] pci-host-generic 3000.vpci: ECAM at [mem > > > 0x3000-0x300f] for [bus 00] > > > [ 4600.669789] pci-host-generic 3000.vpci: PCI host bridge to bus > > > 0001:00 > > > [ 4600.676714] pci_bus 0001:00: root bus resource [bus 00] > > > [ 4600.686212] pci_bus 0001:00: root bus resource [mem > > > 0x3010-0x30101fff] > > > [ 4600.693153] pci 0001:00:00.0: [1af4:1110] type 00 class 0xff > > > [ 4600.693189] pci 0001:00:00.0: reg 0x10: [mem 0x-0x00ff > > > 64bit] > > > [ 4600.693554] PCI: bus0: Fast back to back transfers disabled > > > [ 4600.699246] pci 0001:00:00.0: BAR 0: assigned [mem > > > 0x3010-0x301000ff 64bit] > > > [ 4600.706908] virtio-pci 0001:00:00.0: enabling device ( -> 0002) > > > [ 4600.713576] uio_ivshmem 0001:00:00.0: using jailhouse mode > > > [ 4600.72] The Jailhouse is opening. > > > [ 4601.330640] Created Jailhouse cell "AM572X-IDK-ICSS" > > > > > > > > > > > > > > > # lspci -v > > > > > > 0001:00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory > > > Subsystem: Red Hat, Inc Inter-VM shared memory > > > Flags: bus master, fast devsel, latency 0, IRQ 180 > > > Memory at 3010 (64-bit, non-prefetchable) [size=256] > > > Kernel driver in use: uio_ivshmem > > > > > > > > > > > > > > > # cat /proc/iomem > > > 20013000-2fff : MEM > > > 2010-201f : :00:00.0 > > > 2020-202f : PCI Bus :01 > > > 2020-20203fff : :01:00.0 > > > 2020-20203fff : ec_r8169 > > > 20204000-20204fff : :01:00.0 > > > 20204000-20204fff : ec_r8169 > > > 3000-300f : PCI ECAM > > > 3010-30101fff : //vpci@0 > > > 3010-301000ff : 0001:00:00.0 > > > 3010-301000ff : ivshmem > > > > > > > > > > > > > > > > > > > > > # cat /proc/interrupts > > > > > > > > > 180: 0 GICv2 100 Edge uio_ivshmem > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If I understand correctly, the doorbell address is at 0x3010 + 12 > > > > > > > > > # cat /sys/class/uio/uio0/maps/map0/addr > > > 0x3010 > > > > > > # cat /sys/class/uio/uio0/maps/map1/addr > > > 0xeee0 > > > > > > > > > > > > > > > Whenever I attempt to read or write the doorbell, doesn't matter from > > > which cell, the core gets stuck. > > > On the RTOS side the core gets stuck with no message after trying > > > something like > > > *(0x301c) = 1 > > > > > > > > > > > > > > > On the Linux side after mmap 0x3010 and attempt to write 1 > > > > > > > > > I get > > > > > > > > > FATAL: Invalid ivshmem register read, number 01 > > > FATAL: forbidden access (exception class 0x24) > > > pc=0x0001071e cpsr=0x80040030 hsr=0x91030006 > > > r0=0x0004 r1=0x r2=0xb6ff1000 r3=0xb6ff1001 > > > r4=0xbe80ac18 r5=0x r6=0x r7=0xbe80abd0 > > > r8=0x r9=0x r10=0xb6ff5000 r11=0x > > > r1
Re: ARMv7: using uio_ivshmem to send interrupts between cells
On Monday, April 1, 2019 at 2:18:07 PM UTC+3, Nir Geller wrote:
> Hi there,
>
>
>
>
> I'm developing over ARMv7 Sitara IDK am5728
>
>
> I'm using the jailhouse hypervisor v0.8 and I'm able to run Linux in a root
> cell on core 0 and a TI RTOS as an inmate on core 1.
>
>
> # jailhouse --version
> Jailhouse management tool v0.8
>
>
> A shared memory region seems to be working fine.
> I've set up uio_ivshmem and now I'm trying to send interrupts between cells.
>
>
> [ 4600.643916] OF: PCI: host bridge /vpci@0 ranges:
> [ 4600.648578] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010
> [ 4600.654583] pci-host-generic 3000.vpci: ECAM at [mem
> 0x3000-0x300f] for [bus 00]
> [ 4600.669789] pci-host-generic 3000.vpci: PCI host bridge to bus 0001:00
> [ 4600.676714] pci_bus 0001:00: root bus resource [bus 00]
> [ 4600.686212] pci_bus 0001:00: root bus resource [mem 0x3010-0x30101fff]
> [ 4600.693153] pci 0001:00:00.0: [1af4:1110] type 00 class 0xff
> [ 4600.693189] pci 0001:00:00.0: reg 0x10: [mem 0x-0x00ff 64bit]
> [ 4600.693554] PCI: bus0: Fast back to back transfers disabled
> [ 4600.699246] pci 0001:00:00.0: BAR 0: assigned [mem 0x3010-0x301000ff
> 64bit]
> [ 4600.706908] virtio-pci 0001:00:00.0: enabling device ( -> 0002)
> [ 4600.713576] uio_ivshmem 0001:00:00.0: using jailhouse mode
> [ 4600.72] The Jailhouse is opening.
> [ 4601.330640] Created Jailhouse cell "AM572X-IDK-ICSS"
>
>
>
>
> # lspci -v
>
> 0001:00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
> Subsystem: Red Hat, Inc Inter-VM shared memory
> Flags: bus master, fast devsel, latency 0, IRQ 180
> Memory at 3010 (64-bit, non-prefetchable) [size=256]
> Kernel driver in use: uio_ivshmem
>
>
>
>
> # cat /proc/iomem
> 20013000-2fff : MEM
> 2010-201f : :00:00.0
> 2020-202f : PCI Bus :01
> 2020-20203fff : :01:00.0
> 2020-20203fff : ec_r8169
> 20204000-20204fff : :01:00.0
> 20204000-20204fff : ec_r8169
> 3000-300f : PCI ECAM
> 3010-30101fff : //vpci@0
> 3010-301000ff : 0001:00:00.0
> 3010-301000ff : ivshmem
>
>
>
>
>
>
> # cat /proc/interrupts
>
>
> 180: 0 GICv2 100 Edge uio_ivshmem
>
>
>
>
>
>
>
>
>
> If I understand correctly, the doorbell address is at 0x3010 + 12
>
>
> # cat /sys/class/uio/uio0/maps/map0/addr
> 0x3010
>
> # cat /sys/class/uio/uio0/maps/map1/addr
> 0xeee0
>
>
>
>
> Whenever I attempt to read or write the doorbell, doesn't matter from which
> cell, the core gets stuck.
> On the RTOS side the core gets stuck with no message after trying something
> like
> *(0x301c) = 1
>
>
>
>
> On the Linux side after mmap 0x3010 and attempt to write 1
>
>
> I get
>
>
> FATAL: Invalid ivshmem register read, number 01
> FATAL: forbidden access (exception class 0x24)
> pc=0x0001071e cpsr=0x80040030 hsr=0x91030006
> r0=0x0004 r1=0x r2=0xb6ff1000 r3=0xb6ff1001
> r4=0xbe80ac18 r5=0x r6=0x r7=0xbe80abd0
> r8=0x r9=0x r10=0xb6ff5000 r11=0x
> r12=0x r13=0xbe80abc8 r14=0x0001072f
> Parking CPU 0 (Cell: "AM57XX-EVM")
>
>
>
>
> I also tried uio_send and uio_read but it seems like nothing happens.
>
>
> Is there something wrong with the cell configuration?
> What should I be looking for?
>
>
> Thanks a lot,
>
>
> Nir.
Hi João,
Thanks for your reply.
I'm not a RTOS engineer, my focus is on Linux.
What do you mean by map_range()?
The RTOS project has a configuration file that enables memory regions.
(attached).
I added 0x3000 to the configuration but the behavior is the same.
Mmu.setSecondLevelDescMeta(0x3000, 0x3000, com_attrs);
By default the range is 2MB.
After adding this the RTOS app still gets stuck when trying to peek 0x3010
In an attempt to enable this memory in the inmate cell I tried remapping
0x3010 by adding to the config file:
/* IVSHMEM registers for PCI 0001:00:00.0 */ {
.phys_start = 0x3010,
.virt_start = 0x8cfce000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
},
And indeed the RTOS app doesn't get stuck when trying to read
ARMv7: using uio_ivshmem to send interrupts between cells
Hi there,
I'm developing over ARMv7 Sitara IDK am5728
I'm using the jailhouse hypervisor v0.8 and I'm able to run Linux in a root
cell on core 0 and a TI RTOS as an inmate on core 1.
# jailhouse --version
Jailhouse management tool v0.8
A shared memory region seems to be working fine.
I've set up uio_ivshmem and now I'm trying to send interrupts between cells.
[ 4600.643916] OF: PCI: host bridge /vpci@0 ranges:
[ 4600.648578] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010
[ 4600.654583] pci-host-generic 3000.vpci: ECAM at [mem
0x3000-0x300f] for [bus 00]
[ 4600.669789] pci-host-generic 3000.vpci: PCI host bridge to bus
0001:00
[ 4600.676714] pci_bus 0001:00: root bus resource [bus 00]
[ 4600.686212] pci_bus 0001:00: root bus resource [mem
0x3010-0x30101fff]
[ 4600.693153] pci 0001:00:00.0: [1af4:1110] type 00 class 0xff
[ 4600.693189] pci 0001:00:00.0: reg 0x10: [mem 0x-0x00ff 64bit]
[ 4600.693554] PCI: bus0: Fast back to back transfers disabled
[ 4600.699246] pci 0001:00:00.0: BAR 0: assigned [mem 0x3010-0x301000ff
64bit]
[ 4600.706908] virtio-pci 0001:00:00.0: enabling device ( -> 0002)
[ 4600.713576] uio_ivshmem 0001:00:00.0: using jailhouse mode
[ 4600.72] The Jailhouse is opening.
[ 4601.330640] Created Jailhouse cell "AM572X-IDK-ICSS"
# lspci -v
0001:00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
Subsystem: Red Hat, Inc Inter-VM shared memory
Flags: bus master, fast devsel, latency 0, IRQ 180
Memory at 3010 (64-bit, non-prefetchable) [size=256]
Kernel driver in use: uio_ivshmem
# cat /proc/iomem
20013000-2fff : MEM
2010-201f : :00:00.0
2020-202f : PCI Bus :01
2020-20203fff : :01:00.0
2020-20203fff : ec_r8169
20204000-20204fff : :01:00.0
20204000-20204fff : ec_r8169
3000-300f : PCI ECAM
3010-30101fff : //vpci@0
3010-301000ff : 0001:00:00.0
3010-301000ff : ivshmem
# cat /proc/interrupts
180: 0 GICv2 100 Edge uio_ivshmem
If I understand correctly, the doorbell address is at 0x3010 + 12
# cat /sys/class/uio/uio0/maps/map0/addr
0x3010
# cat /sys/class/uio/uio0/maps/map1/addr
0xeee0
Whenever I attempt to read or write the doorbell, doesn't matter from which
cell, the core gets stuck.
On the RTOS side the core gets stuck with no message after trying something
like
*(0x301c) = 1
On the Linux side after mmap 0x3010 and attempt to write 1
I get
FATAL: Invalid ivshmem register read, number 01
FATAL: forbidden access (exception class 0x24)
pc=0x0001071e cpsr=0x80040030 hsr=0x91030006
r0=0x0004 r1=0x r2=0xb6ff1000 r3=0xb6ff1001
r4=0xbe80ac18 r5=0x r6=0x r7=0xbe80abd0
r8=0x r9=0x r10=0xb6ff5000 r11=0x
r12=0x r13=0xbe80abc8 r14=0x0001072f
Parking CPU 0 (Cell: "AM57XX-EVM")
I also tried uio_send and uio_read but it seems like nothing happens.
Is there something wrong with the cell configuration?
What should I be looking for?
Thanks a lot,
Nir.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
/*
* Jailhouse, a Linux-based partitioning hypervisor
*
* Configuration for uart-demo inmate on AM57XX-EVM:
*
* Copyright (c) Texas Instruments, Inc.
*
* Authors:
* Vitaly Andrianiov
*
* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.
*/
#include
#include
#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])
struct {
struct jailhouse_cell_desc cell;
__u64 cpus[1];
struct jailhouse_memory mem_regions[10];
struct jailhouse_irqchip irqchips[1];
struct jailhouse_pci_device pci_devices[1];
} __attribute__((packed)) config = {
.cell = {
.signature = JAILHOUSE_CELL_DESC_SIGNATURE,
.revision = JAILHOUSE_CONFIG_REVISION,
.name = "AM572X-IDK-ICSS",
.flags = JAILHOUSE_CELL_PASSIVE_COMMREG,
.num_irqchips = 1,
.cpu_set_size = sizeof(config.cpus),
.num_memory_regions = ARRAY_SIZE(config.mem_regions),
.num_pci_devices = ARRAY_SIZE(config.pci_devices),
.vpci_irq_base = 134 - 32,
},
.cpus = {
0x2,
},
.mem_regions = {
/* OCRAM buffer area */ {
.phys_start = 0x4034,
.virt_start = 0x4034,
.size = 0xC,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO | JAILH
Re: Setting up Communication over PCI device
On Wednesday, October 3, 2018 at 10:21:25 PM UTC+3, Jan Kiszka wrote:
> On 03.10.18 12:26, Nir Geller wrote:
> >
> > Hi Jan,
> >
> > After setting
> > num_msix_vectors = 0
> > in the root cell and inmate cell configuration I get good output when
> > starting up the inmate cell:
> >
> > Initializing Jailhouse hypervisor v0.7 (220-g2ad429b) on CPU 0
> > Code location: 0xf040
> > Page pool usage after early setup: mem 30/4072, remap 32/131072
> > Initializing processors:
> > CPU 0... OK
> > CPU 1... OK
> > Adding virtual PCI device 00:00.0 to cell "AM57XX-EVM"
> > Page pool usage after late setup: mem 43/4072, remap 38/131072
> > Activating hypervisor
> > [ 83.501224] OF: PCI: host bridge /vpci@0 ranges:
> > [ 83.505887] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010
> > [ 83.514707] pci-host-generic 3000.vpci: ECAM at [mem
> > 0x3000-0x300f] for [bus 00]
> > [ 83.526709] pci-host-generic 3000.vpci: PCI host bridge to bus
> > 0001:00
> > [ 83.535943] pci_bus 0001:00: root bus resource [bus 00]
> > [ 83.543374] pci_bus 0001:00: root bus resource [mem
> > 0x3010-0x30101fff]
> > [ 83.552696] PCI: bus0: Fast back to back transfers disabled
> > [ 83.563731] pci 0001:00:00.0: BAR 0: assigned [mem 0x3010-0x301000ff
> > 64bit]
> > [ 83.573755] virtio-pci 0001:00:00.0: enabling device ( -> 0002)
> > [ 83.583232] uio_ivshmem 0001:00:00.0: using jailhouse mode
> > [ 83.591699] The Jailhouse is opening.
> > Adding virtual PCI device 00:00.0 to cell "AM572X-IDK-ICSS"
> > Shared memory connection established: "AM572X-IDK-ICSS" <--> "AM57XX-EVM"
> > Created cell "AM572X-IDK-ICSS"
> > Page pool usage after cell creation: mem 63/4072, remap 38/131072
> > [ 84.210960] Created Jailhouse cell "AM572X-IDK-ICSS"
> >
> > root@am57xx-evm:~# lspci -v
> > ...
> > 0001:00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
> > Subsystem: Red Hat, Inc Inter-VM shared memory
> > Flags: bus master, fast devsel, latency 0, IRQ 180
> > Memory at 3010 (64-bit, non-prefetchable) [size=256]
> > Kernel driver in use: uio_ivshmem
> >
> >
> > root@am57xx-evm:~# cat /proc/interrupts
> > CPU0
> > 17: 0 CBAR 32 Level gp_timer
> > 18: 0 GICv2 29 Level arch_timer
> > ...
> > 180: 0 GICv2 32 Edge uio_ivshmem
> > ...
> >
> >
> > Though I couldn't fire the interrupt from any cell.
> >
> > From the linux root cell I tried mmaping 0x3010, then writing 1 to the
> > mapped address+3.
> > I also tried using henning-schild-work/ivshmem-guest-code send_uio to
> > address /dev/uio0 with no success.
> >
> > root@am57xx-evm:~# ./uio_send /dev/uio0 1 1 0
> > [UIO] opening file /dev/uio0
> > [UIO] count is 1
> > [UIO] writing 1
> > [UIO] ping #0
> > [UIO] Exiting...
> > root@am57xx-evm:~#
> >
> > From the RTOS inmate I tried writing 1 to 0x301c, which seems to get
> > the cell stuck without any
> > error message from the hypervisor.
> >
> > The current cell configurations are attached.
>
> [...]
>
> > .root_cell = {
> > .name = "AM57XX-EVM",
> > .cpu_set_size = sizeof(config.cpus),
> > .num_memory_regions = ARRAY_SIZE(config.mem_regions),
> > .num_irqchips = ARRAY_SIZE(config.irqchips),
> > .num_pci_devices = ARRAY_SIZE(config.pci_devices),
> > /*.vpci_irq_base = 96,*/
> > },
>
> You need to set .vpci_irq_base now, in both cells. You need a range of 4
> consecutive GIC interrupts that are not in use by a physical devices. Each
> cell
> should get its own range, though. In the configs for an AM57xx device I have
> lying around here, we were using 96 for the root cell as well and 68 for the
> non-root cell. If you non-root cell uses a device tree as well, make sure
> that
> this number is entered there, too. Otherwise, 68 + % 3
> is
> the SPI number the ivshmem device will get on that side, which you have to
> hard-code into your RTOS.
>
> Jan
Hi Jan,
First thing, I'm trying to send an interrupt from the baremetal inmate to the
root-cell Linux. Later I will try the other way around.
In the dts file some GIC inputs are reserved for the inm
Re: Setting up Communication over PCI device
Hi Jan,
After setting
num_msix_vectors = 0
in the root cell and inmate cell configuration I get good output when starting
up the inmate cell:
Initializing Jailhouse hypervisor v0.7 (220-g2ad429b) on CPU 0
Code location: 0xf040
Page pool usage after early setup: mem 30/4072, remap 32/131072
Initializing processors:
CPU 0... OK
CPU 1... OK
Adding virtual PCI device 00:00.0 to cell "AM57XX-EVM"
Page pool usage after late setup: mem 43/4072, remap 38/131072
Activating hypervisor
[ 83.501224] OF: PCI: host bridge /vpci@0 ranges:
[ 83.505887] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010
[ 83.514707] pci-host-generic 3000.vpci: ECAM at [mem
0x3000-0x300f] for [bus 00]
[ 83.526709] pci-host-generic 3000.vpci: PCI host bridge to bus 0001:00
[ 83.535943] pci_bus 0001:00: root bus resource [bus 00]
[ 83.543374] pci_bus 0001:00: root bus resource [mem 0x3010-0x30101fff]
[ 83.552696] PCI: bus0: Fast back to back transfers disabled
[ 83.563731] pci 0001:00:00.0: BAR 0: assigned [mem 0x3010-0x301000ff
64bit]
[ 83.573755] virtio-pci 0001:00:00.0: enabling device ( -> 0002)
[ 83.583232] uio_ivshmem 0001:00:00.0: using jailhouse mode
[ 83.591699] The Jailhouse is opening.
Adding virtual PCI device 00:00.0 to cell "AM572X-IDK-ICSS"
Shared memory connection established: "AM572X-IDK-ICSS" <--> "AM57XX-EVM"
Created cell "AM572X-IDK-ICSS"
Page pool usage after cell creation: mem 63/4072, remap 38/131072
[ 84.210960] Created Jailhouse cell "AM572X-IDK-ICSS"
root@am57xx-evm:~# lspci -v
...
0001:00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
Subsystem: Red Hat, Inc Inter-VM shared memory
Flags: bus master, fast devsel, latency 0, IRQ 180
Memory at 3010 (64-bit, non-prefetchable) [size=256]
Kernel driver in use: uio_ivshmem
root@am57xx-evm:~# cat /proc/interrupts
CPU0
17: 0 CBAR 32 Level gp_timer
18: 0 GICv2 29 Level arch_timer
...
180: 0 GICv2 32 Edge uio_ivshmem
...
Though I couldn't fire the interrupt from any cell.
>From the linux root cell I tried mmaping 0x3010, then writing 1 to the
>mapped address+3.
I also tried using henning-schild-work/ivshmem-guest-code send_uio to address
/dev/uio0 with no success.
root@am57xx-evm:~# ./uio_send /dev/uio0 1 1 0
[UIO] opening file /dev/uio0
[UIO] count is 1
[UIO] writing 1
[UIO] ping #0
[UIO] Exiting...
root@am57xx-evm:~#
>From the RTOS inmate I tried writing 1 to 0x301c, which seems to get the
>cell stuck without any
error message from the hypervisor.
The current cell configurations are attached.
Thanks a lot,
Nir.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
/*
* Jailhouse, a Linux-based partitioning hypervisor
*
* Configuration for uart-demo inmate on AM57XX-EVM:
*
* Copyright (c) Texas Instruments, Inc.
*
* Authors:
* Vitaly Andrianiov
*
* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.
*/
#include
#include
#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])
struct {
struct jailhouse_cell_desc cell;
__u64 cpus[1];
struct jailhouse_memory mem_regions[9];
struct jailhouse_irqchip irqchips[1];
struct jailhouse_pci_device pci_devices[1];
} __attribute__((packed)) config = {
.cell = {
.signature = JAILHOUSE_CELL_DESC_SIGNATURE,
.revision = JAILHOUSE_CONFIG_REVISION,
.name = "AM572X-IDK-ICSS",
.flags = JAILHOUSE_CELL_PASSIVE_COMMREG,
.num_irqchips = 1,
.cpu_set_size = sizeof(config.cpus),
.num_memory_regions = ARRAY_SIZE(config.mem_regions),
.num_pci_devices = ARRAY_SIZE(config.pci_devices),
/*.vpci_irq_base = 96,*/
},
.cpus = {
0x2,
},
.mem_regions = {
/* OCRAM buffer area */ {
.phys_start = 0x4034,
.virt_start = 0x4034,
.size = 0xC,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
},
/* UART... */ {
.phys_start = 0x4802,
.virt_start = 0x4802,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
},
/* ... */ {
.phys_start = 0x4805,
.virt_start = 0x4805,
.size = 0x1,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
},
{
/* TIMER8 */
.phys_start = 0x48826000,
.virt_start = 0x48826000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
},
/* L4_CFG */ {
.phys_start = 0x4a00,
.virt_start = 0x4a00,
.size = 0xE1,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE
Re: Setting up Communication over PCI device
0 0 48051000.gpio 12 Edge tpd12s015 hpd
173: 0 0 4805d000.gpio 27 Edge 4809c000.mmc cd
174: 0 0 48057000.gpio 16 Edge palmas_usb_id
176: 0 0 48057000.gpio 26 Edge palmas_usb_vbus
177: 0 0 CBAR 71 Level xhci-hcd:usb1
IPI0: 0 1 CPU wakeup interrupts
IPI1: 0 0 Timer broadcast interrupts
IPI2: 21693 26808 Rescheduling interrupts
IPI3:356619 Function call interrupts
IPI4: 0 0 CPU stop interrupts
IPI5: 0 0 IRQ work interrupts
IPI6: 0 0 completion interrupts
Err:
Thanks a lot,
Nir.
On Tue, Sep 18, 2018 at 11:12 AM Jan Kiszka wrote:
> On 18.09.18 10:06, Jan Kiszka wrote:
> > On 18.09.18 09:57, Nir Geller wrote:
> >> Thanks for your reply.
> >>
> >> I recompiled the kernel.
> >>
> >> Now when I try to enable the root cell I get this:
> >>
> >> [ 95.597454] OF: PCI: host bridge /vpci@0 ranges:
> >> [ 95.602141] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010
> >> [ 95.608155] pci-host-generic 3000.vpci: ECAM at [mem
> >> 0x3000-0x300f] for [bus 00]
> >> [ 95.622972] pci-host-generic 3000.vpci: Node /vpci@0 has
> inconsistent
> >> "linux,pci-domain" property in DT
> >
> > Either your jailhouse system config needs to set .pci_domain for the
> virtual
> > host controller to 1, or you are missing linux,pci-domain in the device
> tree
> > entry of the physical controller of that target. Both device tree
> entries need
> > to be consistent /wrt this property, i.e. both use it or both leave it
> out,
> > that's what Linux means here.
> >
>
> Oh, and the recommended variant is using linux,pci-domain because it gives
> a
> stable domain number across jailhouse disable / enable cycles. We are
> slowly
> converting boards (by adding patches for device trees that lack the entry).
>
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux
>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
/*
* Jailhouse, a Linux-based partitioning hypervisor
*
* Copyright (c) Texas Insturments. Inc, 2016
*
* Authors:
* Vitaly Andrianov
*
* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.
*/
#include
#include
#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])
struct {
struct jailhouse_system header;
__u64 cpus[1];
struct jailhouse_memory mem_regions[19];
struct jailhouse_irqchip irqchips[2];
struct jailhouse_pci_device pci_devices[1];
} __attribute__((packed)) config = {
.header = {
.signature = JAILHOUSE_SYSTEM_SIGNATURE,
.revision = JAILHOUSE_CONFIG_REVISION,
.hypervisor_memory = {
.phys_start = 0xef00,
.size = 0x100,
},
.debug_console = {
.address = 0x4802,
.size = 0x1000,
/* .divider = 26, */
.flags = JAILHOUSE_CON1_TYPE_8250 |
JAILHOUSE_CON1_ACCESS_MMIO |
JAILHOUSE_CON1_REGDIST_4 |
JAILHOUSE_CON2_TYPE_ROOTPAGE,
},
.platform_info = {
.pci_mmconfig_base = 0x3000,
.pci_mmconfig_end_bus = 0,
.pci_is_virtual = 1,
.arm = {
.gic_version = 2,
.gicd_base = 0x48211000,
.gicc_base = 0x48212000,
.gich_base = 0x48214000,
.gicv_base = 0x48216000,
.maintenance_irq = 25,
},
},
.root_cell = {
.name = "AM57XX-EVM",
.cpu_set_size = sizeof(config.cpus),
.num_memory_regions = ARRAY_SIZE(config.mem_regions),
.num_irqchips = ARRAY_SIZE(config.irqchips),
.num_pci_devices = ARRAY_SIZE(config.pci_devices),
.vpci_irq_base = 134,
},
},
.cpus = {
0x3,
},
.mem_regions = {
/* PCI */ {
.phys_start = 0x2000,
.virt_start = 0x2000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
/* OCMCRAM */ {
.phys_start = 0x4030,
.virt_start = 0x4030,
.size = 0x8,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
/* 0x4038 - 0x4802 */ {
.phys_start = 0x4038,
.virt_start = 0x4038,
.size = 0x7ca,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
/* UART... */ {
.phys_start = 0x4802,
.virt_start = 0x4802,
.size = 0xe,//0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
/* 0x4810 - 0x48281000 */ {
.phys_start = 0x
Re: Setting up Communication over PCI device
Thanks for your reply. I recompiled the kernel. Now when I try to enable the root cell I get this: [ 95.597454] OF: PCI: host bridge /vpci@0 ranges: [ 95.602141] OF: PCI: MEM 0x3010..0x30101fff -> 0x3010 [ 95.608155] pci-host-generic 3000.vpci: ECAM at [mem 0x3000-0x300f] for [bus 00] [ 95.622972] pci-host-generic 3000.vpci: Node /vpci@0 has inconsistent "linux,pci-domain" property in DT [ 95.632989] pci-host-generic 3000.vpci: PCI host bridge to bus :00 [ 95.640309] pci_bus :00: root bus resource [bus 00] [ 95.645915] pci_bus :00: root bus resource [mem 0x3010-0x30101fff] [ 95.653206] pci :00:02.0: [1af4:1110] type 00 class 0xff [ 95.653243] pci :00:02.0: reg 0x10: [mem 0x-0x00ff 64bit] [ 95.653632] PCI: bus0: Fast back to back transfers disabled [ 95.659250] pci :00:02.0: BAR 0: assigned [mem 0x3010-0x301000ff 64bit] [ 95.667132] virtio-pci :00:02.0: enabling device ( -> 0002) root@am57xx-evm:~# lspci -v lspci: sysfs_scan: Invalid domain root@am57xx-evm:~# root@am57xx-evm:~# cat /proc/iomem 20013000-2fff : MEM 2010-201f : :00:00.0 2020-202f : PCI Bus :01 2020-20203fff : :01:00.0 2020-20203fff : ec_r8169 20204000-20204fff : :01:00.0 20204000-20204fff : ec_r8169 3000-300f : PCI ECAM 3010-30101fff : //vpci@0 3010-301000ff : :00:02.0 ... Is the problem related to bar_mask config? How can I fix this? What value should be assigned to .vpci_irq_base? Thanks, Nir. On Tue, Sep 18, 2018 at 8:11 AM Jan Kiszka wrote: > On 17.09.18 17:04, Nir Geller wrote: > > > > Hi there, > > > > > > I'm developing over Sitara IDK am5728 with TI supplied > PROCESSOR-SDK-LINUX-AM57X > > version v05.00 and PROCESSOR-SDK-RTOS-AM57X version v05.00. > > > > I'm using the jailhouse hypervisor and I'm able to run Linux in a root > cell on > > core 0 and a TI RTOS as an inmate on core 1. > > > > Now I'm trying to set up a pci device to allow communication between the > two cores. > > I have a realtek etheret adapter mounted on the PCI-E slot, and it is > being used > > by Linux. > > > > After going through the examples and reading posts in the forum I > adapted the > > root cell configuration (pci_am57xx-evm.c attached). > > > > I didn't understand how to figure out the .pci_mmconfig_base address. > > In case of a virtual PCI host controller, like you configured, this > address is > just some free range in the physical address space of your board. > > > > > When enabling the root cell I get a CONFIG_OF_OVERLAY error message, and > no PCI > > device appears in lspci -v: > > > > root@am57xx-evm:~# modprobe jailhouse > > root@am57xx-evm:~# jailhouse enable jail/pci_am57xx-evm.cell > > > > Initializing Jailhouse hypervisor v0.7 (220-g2ad429b) on CPU 0 > > Code location: 0xf040 > > Page pool usage after early setup: mem 30/4072, remap 32/131072 > > Initializing processors: > > CPU 0... OK > > CPU 1... OK > > Adding virtual PCI device 00:01.0 to cell "AM57XX-EVM" > > Page pool usage after late setup: mem 43/4072, remap 38/131072 > > Activating hypervisor > > [ 857.467894] jailhouse: CONFIG_OF_OVERLAY disabled > > [ 857.472645] jailhouse: failed to add virtual host controller > > [ 857.478334] The Jailhouse is opening. > > root@am57xx-evm:~# > > root@am57xx-evm:~# lspci -v > > 00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC > (rev 01) > > (prog-if 00 [Normal decode]) > > Flags: bus master, fast devsel, latency 0, IRQ 170 > > Memory at 2010 (64-bit, non-prefetchable) [size=1M] > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > I/O behind bridge: 1000-1fff [size=4K] > > Memory behind bridge: None > > Prefetchable memory behind bridge: 2020-202f [size=1M] > > Capabilities: [40] Power Management version 3 > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ > > Capabilities: [70] Express Root Port (Slot-), MSI 00 > > Capabilities: [100] Advanced Error Reporting > > Kernel driver in use: pcieport > > > > 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. > RTL8111/8168/8411 > > PCI Express Gigabit Ethernet Controller (rev 06) > > Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 > PCI > > Express Gigabit Ethernet Controller > > Flags
Re: Porting RTOS project to inmate
On Thursday, September 6, 2018 at 12:05:06 PM UTC+3, Nir Geller wrote:
> Hi there,
>
> I'm developing over Sitara IDK am5728 with TI supplied
> PROCESSOR-SDK-LINUX-AM57X version v05.00 and PROCESSOR-SDK-RTOS-AM57X version
> v05.00.
> I'm using the jailhouse hypervisor and I'm able to run linux on core 0 and on
> core 1 the prebuilt jailhouse inmate demo taken from
> processor_sdk_rtos_am57xx_5_00_00_15/demos/jailhouse-inmate/rtos/icss_emac
> # modprobe jailhouse
> # jailhouse enable /home/root/jail/am57xx-evm.cell
> # jailhouse cell create /usr/share/jailhouse/examples/am572x-rtos-icss.cell
> # jailhouse cell load 1 /usr/share/jailhouse/examples/linux-loader.bin -a 0
> -s "kernel=0x8000" -a 0x100 /home/root/jail/icss_emac.bin -a 0x8000
> # jailhouse cell start 1
> When starting the cell I get outputs over UART.
> I can connect over JTAG and see that the memory browser and disassembly show
> exactly what is expected
> according to objdumbing icss_emac.out.
> Now I'm trying to build my own RTOS inmate using PROCESSOR-SDK-RTOS-AM57X
> version v05.00.
> Since I was able to run successfully the icss_emac jailhouse demo I took
> pdk_am57xx_1_0_11/packages/MyExampleProjects/ICSS_EMAC_BasicExample_idkAM572x_wSoCLib_armTestproject
> as a starting point. Before making any changes, when I ran this project using
> JTAG it was working fine and I got its output over UART.
> Now I'm trying to port it into a jailhouse inmate.
> So I followed
> processors.wiki.ti.com/.../Processor_SDK_Jailhouse_Hypervisor
> From what I gather there are 3 main differences between the jailhouse inmate
> and the original sys/bios project:
> compilation flag __JAILHOUSE_INMATE
> board init function
> cfg file
> I tried all kinds of variations like taking the cfg and board_init files from
> demos/jailhouse-inmate/rtos/icss_emac/src and add to
> ICSS_EMAC_BasicExample_idkAM572x_wSoCLib_armTestproject.
> I added the compilation flag __JAILHOUSE_INMATE
> but so far I wasn't able to create a .bin file that I could start up as an
> inmate on core 1.
> I created an inmate cell with:
> # modprobe jailhouse
> # jailhouse enable /home/root/jail/am57xx-evm.cell
> # jailhouse cell create /usr/share/jailhouse/examples/am572x-rtos-icss.cell
> Copied the app product from
> ICSS_EMAC_BasicExample_idkAM572x_wSoCLib_armTestproject and loaded it.
> By objdumping the .out I found that the entry point is 0x80001000
> # jailhouse cell load 1 /usr/share/jailhouse/examples/linux-loader.bin -a 0
> -s "kernel=0x80001000" -a 0x100 /home/root/jail/app -a 0x8000
> # jailhouse cell start 1
> At this point I get no output over UART.
> The disassembly of .out (attached) looks like
> Disassembly of section .c_int00:
> 80001000 <_c_int00>:
> 80001000: e10f mrs r0, CPSR
> 80001004: e380c0c0 orr ip, r0, #192 ; 0xc0
>
>
> Disassembly of section .text:
> 80001590 :
> 80001590: e16d41f0 strd r4, [sp, #-16]!
> 80001594: e58d6008 str r6, [sp, #8]
> 80001598: e58de00c str lr, [sp, #12]
>
>
>
> When using JTAG to inspect the memory it seems like the file was loaded to
> 0x8048
> And the text section starts from 0x81Ec (see attached picture)
>
> What can I do to start up this project as a jailhouse inmate?
> Any assist would be great.
>
> Thanks a lot,
>
> Nir.
Attached is an example of a configuration file that allows an RTOS project to
run as an inmate, and a substitute function to init the board.
The basic idea is to prevent any board_init procedures, as they were already
executed by Linux, and to create a bin file that uses contiguous memory
starting from 0x8000.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
var Defaults = xdc.useModule('xdc.runtime.Defaults');
var Diags = xdc.useModule('xdc.runtime.Diags');
var Error = xdc.useModule('xdc.runtime.Error');
var Log = xdc.useModule('xdc.runtime.Log');
var LoggerBuf = xdc.useModule('xdc.runtime.LoggerBuf');
var Main = xdc.useModule('xdc.runtime.Main');
var Memory = xdc.useModule('xdc.runtime.Memory')
var SysMin = xdc.useModule('xdc.runtime.SysMin');
var System = xdc.useModule('xdc.runtime.System');
var Text = xdc.useModule('xdc.runtime.Text');
var Clock = xdc.useModule('ti.sysbios.knl.Clock');
var Swi = xdc.useModule('ti.sysbios.knl.Swi');
var Task = xdc.useModule('ti.sysbios.knl.Task');
var Semaphore = xdc.useModule(&
Setting up Communication over PCI device
Hi there,
I'm developing over Sitara IDK am5728 with TI supplied
PROCESSOR-SDK-LINUX-AM57X version v05.00 and PROCESSOR-SDK-RTOS-AM57X
version v05.00.
I'm using the jailhouse hypervisor and I'm able to run Linux in a root cell
on core 0 and a TI RTOS as an inmate on core 1.
Now I'm trying to set up a pci device to allow communication between the
two cores.
I have a realtek etheret adapter mounted on the PCI-E slot, and it is being
used by Linux.
After going through the examples and reading posts in the forum I adapted
the root cell configuration (pci_am57xx-evm.c attached).
I didn't understand how to figure out the .pci_mmconfig_base address.
When enabling the root cell I get a CONFIG_OF_OVERLAY error message, and no
PCI device appears in lspci -v:
root@am57xx-evm:~# modprobe jailhouse
root@am57xx-evm:~# jailhouse enable jail/pci_am57xx-evm.cell
Initializing Jailhouse hypervisor v0.7 (220-g2ad429b) on CPU 0
Code location: 0xf040
Page pool usage after early setup: mem 30/4072, remap 32/131072
Initializing processors:
CPU 0... OK
CPU 1... OK
Adding virtual PCI device 00:01.0 to cell "AM57XX-EVM"
Page pool usage after late setup: mem 43/4072, remap 38/131072
Activating hypervisor
[ 857.467894] jailhouse: CONFIG_OF_OVERLAY disabled
[ 857.472645] jailhouse: failed to add virtual host controller
[ 857.478334] The Jailhouse is opening.
root@am57xx-evm:~#
root@am57xx-evm:~# lspci -v
00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC
(rev 01) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0, IRQ 170
Memory at 2010 (64-bit, non-prefetchable) [size=1M]
Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
I/O behind bridge: 1000-1fff [size=4K]
Memory behind bridge: None
Prefetchable memory behind bridge: 2020-202f [size=1M]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Root Port (Slot-), MSI 00
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: pcieport
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI
Express Gigabit Ethernet Controller
Flags: bus master, fast devsel, latency 0, IRQ 179
I/O ports at 1000 [size=256]
Memory at 20204000 (64-bit, prefetchable) [size=4K]
Memory at 2020 (64-bit, prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable- Count=4 Masked-
Capabilities: [d0] Vital Product Data
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel
Capabilities: [160] Device Serial Number 01-00-00-00-68-4c-e0-00
Kernel driver in use: ec_r8169
root@am57xx-evm:~#
How should I fix the root cell config? what should I be looking for?
Thanks a lot,
Nir.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
/*
* Jailhouse, a Linux-based partitioning hypervisor
*
* Copyright (c) Texas Insturments. Inc, 2016
*
* Authors:
* Vitaly Andrianov
*
* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.
*/
#include
#include
#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])
struct {
struct jailhouse_system header;
__u64 cpus[1];
struct jailhouse_memory mem_regions[19];
struct jailhouse_irqchip irqchips[2];
struct jailhouse_pci_device pci_devices[1];
} __attribute__((packed)) config = {
.header = {
.signature = JAILHOUSE_SYSTEM_SIGNATURE,
.revision = JAILHOUSE_CONFIG_REVISION,
.hypervisor_memory = {
.phys_start = 0xef00,
.size = 0x100,
},
.debug_console = {
.address = 0x4802,
.size = 0x1000,
/* .divider = 26, */
.flags = JAILHOUSE_CON1_TYPE_8250 |
JAILHOUSE_CON1_ACCESS_MMIO |
JAILHOUSE_CON1_REGDIST_4 |
JAILHOUSE_CON2_TYPE_ROOTPAGE,
},
.platform_info = {
.pci_mmconfig_base = 0x4000,
.pci_mmconfig_end_bus = 0,
.pci_is_virtual = 1,
Re: PCIe causes fatal errors
Hi Jan,
What eventually seems to be working properly is:
{
.phys_start = 0x2000,
.virt_start = 0x2000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
Thanks a lot,
Nir.
On Sun, Aug 26, 2018 at 8:58 PM Jan Kiszka wrote:
> On 2018-08-26 10:50, Nir Geller wrote:
> >
> > Hi there,
> >
> > I recently started developing a system on a Sitara am5728 by TI.
> > This cpu has 2 arm7 A15 cores.
> >
> > I aim to run Linux on the root cell and RTOS on the inmate cell.
> >
> > At the moment I'm using an evaluation board from TI.
> > This board has 1 PCIe slot, on which a realtek ethernet adapter is
> installed.
> >
> > When I startup Linux without jailhouse I'm able to modprobe the r8169
> driver and use the ethernet adapter without any problem.
> > In my system RTOS has no need to use the PCIe.
> >
> > The problem:
> >
> > After I modprobe jailhouse.ko and create the root cell, I attempt to
> modprobe r8169 and I get this:
> >
> > root@am57xx-evm:~# modprobe jailhouse
> > root@am57xx-evm:~# jailhouse enable
> /usr/share/jailhouse/examples/am57xx-evm.cell
> >
> > Initializing Jailhouse hypervisor v0.7 (220-g2ad429b) on CPU 0
> > Code location: 0xf040
> > Page pool usage after early setup: mem 30/4072, remap 32/131072
> > Initializing processors:
> > CPU 0... OK
> > CPU 1... OK
> > Page pool usage after late setup: mem 40/4072, remap 38/131072
> > Activating hypervisor
> > [ 55.866355] The Jailhouse is opening.
> > root@am57xx-evm:~#
> > root@am57xx-evm:~#
> > root@am57xx-evm:~# modprobe r8169
> > Unhandled data read at 0x2000103d(1)
> > FATAL: unhandled trap (exception class 0x24)
> > pc=0xc05554b8 cpsr=0x60010093 hsr=0x93010005
> > r0=0xe108f03d r1=0x0001 r2=0xd29bfc4c r3=0x
> > r4=0xd393e628 r5=0xe108f000 r6=0x0100 r7=0x003d
> > r8=0x0001 r9=0xd393e610 r10=0x r11=0xd29bfbec
> > r12=0xd29bfbf0 r13=0xd29bfbe0 r14=0xc0556254
> > Parking CPU 0 (Cell: "AM57XX-EVM")
> >
> > So if I understand correctly, jailhouse forbids Linux from reading
> address 0x2000103d and stops core 0.
> >
> > /proc/iomem and /proc/ioports show that PCI address space is indeed
> mapped to the 0x2000 region.
> >
> >
> > root@am57xx-evm:~# dmesg | grep -i pci
> > ...
> > [0.762245] OF: PCI: host bridge /ocp/axi@0/pcie@5100 ranges:
> > [0.762280] OF: PCI:IO 0x20003000..0x20012fff -> 0x
> > [0.762306] OF: PCI: MEM 0x20013000..0x2fff -> 0x20013000
> > ...
> >
> > By looking at am57xx-evm.c I don't see where this address space is
> handled. I tried adding it to .mem_regions but I got a similar result:
>
> That addition is not reflected in the config you attached, is it? But
> that is the way to go. Maybe you only made some mistake. Basically, this
> plus an increase of the mem_regions array by 1 should do it:
>
> {
> .phys_start = 0x20013000,
> .virt_start = 0x20013000,
> .size = 0xffed000,
> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
> JAILHOUSE_MEM_IO,
> },
>
> Vitaly, left out the PCI region intentionally?
>
> Jan
>
>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
PCIe causes fatal errors
Hi there,
I recently started developing a system on a Sitara am5728 by TI.
This cpu has 2 arm7 A15 cores.
I aim to run Linux on the root cell and RTOS on the inmate cell.
At the moment I'm using an evaluation board from TI.
This board has 1 PCIe slot, on which a realtek ethernet adapter is installed.
When I startup Linux without jailhouse I'm able to modprobe the r8169 driver
and use the ethernet adapter without any problem.
In my system RTOS has no need to use the PCIe.
The problem:
After I modprobe jailhouse.ko and create the root cell, I attempt to modprobe
r8169 and I get this:
root@am57xx-evm:~# modprobe jailhouse
root@am57xx-evm:~# jailhouse enable
/usr/share/jailhouse/examples/am57xx-evm.cell
Initializing Jailhouse hypervisor v0.7 (220-g2ad429b) on CPU 0
Code location: 0xf040
Page pool usage after early setup: mem 30/4072, remap 32/131072
Initializing processors:
CPU 0... OK
CPU 1... OK
Page pool usage after late setup: mem 40/4072, remap 38/131072
Activating hypervisor
[ 55.866355] The Jailhouse is opening.
root@am57xx-evm:~#
root@am57xx-evm:~#
root@am57xx-evm:~# modprobe r8169
Unhandled data read at 0x2000103d(1)
FATAL: unhandled trap (exception class 0x24)
pc=0xc05554b8 cpsr=0x60010093 hsr=0x93010005
r0=0xe108f03d r1=0x0001 r2=0xd29bfc4c r3=0x
r4=0xd393e628 r5=0xe108f000 r6=0x0100 r7=0x003d
r8=0x0001 r9=0xd393e610 r10=0x r11=0xd29bfbec
r12=0xd29bfbf0 r13=0xd29bfbe0 r14=0xc0556254
Parking CPU 0 (Cell: "AM57XX-EVM")
So if I understand correctly, jailhouse forbids Linux from reading address
0x2000103d and stops core 0.
/proc/iomem and /proc/ioports show that PCI address space is indeed mapped to
the 0x2000 region.
root@am57xx-evm:~# dmesg | grep -i pci
...
[0.762245] OF: PCI: host bridge /ocp/axi@0/pcie@5100 ranges:
[0.762280] OF: PCI:IO 0x20003000..0x20012fff -> 0x
[0.762306] OF: PCI: MEM 0x20013000..0x2fff -> 0x20013000
...
By looking at am57xx-evm.c I don't see where this address space is handled. I
tried adding it to .mem_regions but I got a similar result:
root@am57xx-evm:~# modprobe r8169
FATAL: Invalid MMIO read, address: 2000103d, size: 1
FATAL: forbidden access (exception class 0x24)
pc=0xc05554b8 cpsr=0x60010093 hsr=0x93010005
r0=0xe108f03d r1=0x0001 r2=0xd2a1dc4c r3=0x
r4=0xd393e628 r5=0xe108f000 r6=0x0100 r7=0x003d
r8=0x0001 r9=0xd393e610 r10=0x r11=0xd2a1dbec
r12=0xd2a1dbf0 r13=0xd2a1dbe0 r14=0xc0556254
Parking CPU 0 (Cell: "AM57XX-EVM")
This time it says "Invalid MMIO read"
I also tried adding a pci_device but I'm not sure if I've done so correctly.
Anyway, the result was the same.
I'm attaching dmesg, iomem, ioports and the root cell configuration.
Hopefully you can help me configure my system correctly and allow jailhouse and
PCI play nicely together.
Thanks a lot,
Nir.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
/*
* Jailhouse, a Linux-based partitioning hypervisor
*
* Copyright (c) Texas Insturments. Inc, 2016
*
* Authors:
* Vitaly Andrianov
*
* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.
*/
#include
#include
#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])
struct {
struct jailhouse_system header;
__u64 cpus[1];
struct jailhouse_memory mem_regions[17];
struct jailhouse_irqchip irqchips[2];
} __attribute__((packed)) config = {
.header = {
.signature = JAILHOUSE_SYSTEM_SIGNATURE,
.revision = JAILHOUSE_CONFIG_REVISION,
.hypervisor_memory = {
.phys_start = 0xef00,
.size = 0x100,
},
.debug_console = {
.address = 0x4802,
.size = 0x1000,
/* .divider = 26, */
.flags = JAILHOUSE_CON1_TYPE_8250 |
JAILHOUSE_CON1_ACCESS_MMIO |
JAILHOUSE_CON1_REGDIST_4 |
JAILHOUSE_CON2_TYPE_ROOTPAGE,
},
.platform_info.arm = {
.gic_version = 2,
.gicd_base = 0x48211000,
.gicc_base = 0x48212000,
.gich_base = 0x48214000,
.gicv_base = 0x48216000,
.maintenance_irq = 25,
},
.root_cell = {
.name = "AM57XX-EVM",
.cpu_set_size = sizeof(config.cpus),
.num_memory_regions = ARRAY_SIZE(config.mem_regions),
.num_irqchips = ARRAY_SIZE(config.irqchips),
},
},
.cpus = {
0x3,
},
.mem_regions = {
/* OCMCRAM */ {
.phys_start = 0x4030,
.virt_start = 0x4030,
.size = 0x8,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
/* 0x4038 - 0x4802 */ {
.phys_start = 0x4038,
.virt_start = 0x4038,
.size = 0x7ca,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
/* UART... */ {
.phys_start = 0x4802,
.virt_start = 0x4802,
.size = 0xe,//0x100
