On 28/01/2021 14:15, Bram Hooimeijer wrote:
>> Hi Bram,
>>
>> On 27/01/2021 11:46, Bram Hooimeijer wrote:
>>> Dear Jailhouse community,
>>>
>>> Currently I am experiencing issues with the MMIO virtualization in
>> Jailhouse, which interacts with the BIOS. I am hoping any of you can shed
>> some light on the issue:
>>>
>>> Jailhouse reports the following after running for some while:
>>>> FATAL: unable to get MMIO instruction
>>
>> Where does RIP point to?
>
> I don't know how to derive this exactly. The issue arrives from root-cell
> linux, so the RIP is not in the cell binary or in jailhouse.
>
>>
>>>
>>> The region listed has previously reported illegal MMIO accesses. These
>> have been resolved by adding the region to the system config. This requires
>> all sub-64 access widths, as the region is apparently byte accessed. The BIOS
>> vendor has indicated that this region implements APEI according to the ACPI
>> specification.
>>
>> Could you please try the following region:
>> {
>> .phys_start = 0x86dff000,
>> .virt_start = 0x86dff000,
>> .size = 0x0b700000,
>> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
>> JAILHOUSE_MEM_EXECUTE |
>> JAILHOUSE_MEM_DMA },
>>
>> the size 0x0b700000 will make the page aligned, and the hypervisor doesn't
>> need to trap. In that case, JH_MEM_IO_{8,16,32,64} is not required.
>
> This seems to solve the issue for now.
Great.
> As I understand: if the region is page-aligned, access is possible without
> involving the hypervisor as it is known up front that this access is the cell
> its own region.
Yes, {phys,virt}_start and size need to be page-aligned, then no
hypervison intervention is required.
> If the region is not page aligned, an access should involve the hypervisor to
> prevent access to the page outside the reserved region.
Exactly, we call that subpaging. You need that if a single physical page
holds, e.g., two devices that need to be assigned to different cells.
The hypervisor will moderate the access, and prevent accesses outside
the cell's scope.
And what happened in your case: The hypervisor needs to decode the
instruction that caused the access. We only support decoding of the most
frequent instructions that are commonly used for MMIO access.
Ralf
>
> Thanks for the help!
> Bram
>
>>
>> Thanks
>> Ralf
>>
>>
>>>
>>> By adding some printk() statements to the hypervisor, I could derive where
>> the error exactly originates. Apparently the function ctx_update(&ctx, &pc,
>> 0, pg_structs) returns null. Jan, could you shed any light on what this
>> function
>> does? Could it be that the paging structs for the APEI interface are
>> incorrect?
>>>
>>> All snippets are listed below. The system runs Linux Kernel 5.4.73, from
>> Ubuntu 20.04 LTS, modified to remove kernel RDT support (conflicts with
>> Jailhouse). Any directions are welcome!
>>>
>>> Many thanks,
>>>
>>> Kind regards,
>>>
>>> Bram Hooimeijer
>>>
>>> Issue reported over ipmi (see attached log.txt):
>>>> FATAL: unable to get MMIO instruction
>>>> FATAL: Invalid MMIO/RAM read, addr: 0x00000000870a9998 size: 0
>>>> RIP: 0xfffffffeec8a9998 RSP: 0xffffb52400177d78 FLAGS: 10282
>>>> RAX: 0x0000000000000246 RBX: 0xffffb5240dc8fe50 RCX:
>>>> 0xffff9cef966f4000
>>>> RDX: 0xffff9cef966f4400 RSI: 0xffff9cef966f4000 RDI:
>>>> 0xfffffffeec8a9998
>>>> CS: 10 BASE: 0x0000000000000000 AR-BYTES: a09b EFER.LMA 1
>>>> CR0: 0x0000000080050033 CR3: 0x0000002ff2842001 CR4:
>>>> 0x00000000007626f0
>>>> EFER: 0x0000000000000d01
>>>> Parking CPU 0 (Cell: "RootCell")
>>>
>>> Modification required to root cell configuration for APEI ERST region
>>> (see attached sysconfig_dual256M.c)
>>>> {
>>>> .phys_start = 0x86dff000,
>>>> .virt_start = 0x86dff000,
>>>> .size = 0x0b6fffff,
>>>> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
>>>> JAILHOUSE_MEM_IO | JAILHOUSE_MEM_IO_UNALIGNED |
>>>> JAILHOUSE_MEM_ROOTSHARED | JAILHOUSE_MEM_IO_8 |
>>>> JAILHOUSE_MEM_IO_16 | JAILHOUSE_MEM_IO_32 |
>>>> JAILHOUSE_MEM_IO_64 | JAILHOUSE_MEM_EXECUTE },
>>>
>>> Point of failure (see hypervisor/arch/x86/mmio.c:108--121)
>>>> struct mmio_instruction x86_mmio_parse(const struct
>>>> guest_paging_structures *pg_structs, bool is_write) {
>>>> struct parse_context ctx = { .remaining = X86_MAX_INST_LEN,
>>>> .count = 1 };
>>>> union registers *guest_regs = &this_cpu_data()->guest_regs;
>>>> struct mmio_instruction inst = { 0 };
>>>> u64 pc = vcpu_vendor_get_rip();
>>>> unsigned int n, skip_len = 0;
>>>> union opcode op[4] = { };
>>>>
>>>> if (!ctx_update(&ctx, &pc, 0, pg_structs)){
>>>> printk("ctx update failed\n");
>> <<--- FAILS HERE.
>>>> goto error_noinst;
>>>> }
>>>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/2b5b52c8-58c1-7368-52f5-9f62385516cf%40oth-regensburg.de.
