Hi,
On 11/01/2024 13:57, Michele Pescapè wrote:
Hi,
after changing the virt_start to match the phys_start for that memory
region I got an invalid argument error when starting the cell
Traceback (most recent call last):
File "/usr/local/libexec/jailhouse/jailhouse-cell-linux", line 739,
in <module>
cell.load(arch.kernel_image, arch.kernel_address())
File "/usr/local/lib/python3.10/dist-packages/pyjailhouse/cell.py",
line 44, in load
fcntl.ioctl(self.dev, self.JAILHOUSE_CELL_LOAD, load)
OSError: [Errno 22] Invalid argument
Are you sure that you created the cell with the correct config? What
does dmesg say? Any output on the hypervisor console when this error
happens?
Printing the arch.kernel_address() returned 16760320 (should be
0xffbe00), so I thought that the bzImage was too big for the low ram region.
jailhouse-cell-linux won't load the bzImage to the low ram region. You
can use the -w option to dump a file that prints the commands that
jailhouse-cell-linux would invoke.
I increased the size to .size = 0x04000000 and shifted the virt and
phys_start of the other two regions to match the change (virt_start =
0x04000000 for the comm region and virt=phys_start = 0x46000000 for the
third region) .
No issues were reported by config check but after starting the cell i
got no output even using earlyprintk.
I also tried using an initrd but I got a similar issue, invalid argument
when starting the cell, this time it was the arch.ramdisk_address()
which was 107327488 (0x665b000).
So i tried increasing even more the size of the first region to size =
0x0a000000 (comm region now starts at .virt_start = 0x0a000000, and the
third region starts at virt=.phys_start = 0x4c000000 with size =
0x6000000).
Again, no config check issues so I started it but no output again, the
hypervisor only reports:
Created cell "linux-1"
Page pool usage after cell creation: mem 268/32211, remap 16392/131072
Cell "linux-1" can be loaded
CPU 5 received SIPI, vector 100
CPU 4 received SIPI, vector 100
Started cell "linux-1"
My bzImage is 12,9MB and the rootfs.cpio is 23.1MB if that matters.
Try starting without the ramdisk. We can later take care of the ramdisk,
but first the kernel needs to start.
For a compressed image, your kernel is really huge. I bet you can boil
it down to a few MBs. With a kernel of that size, you might want to play
around with jailhouse-cell-linux -k.
Ralf
Thanks,
Michele
Il giorno mercoledì 10 gennaio 2024 alle 20:58:03 UTC+1 Ralf Ramsauer ha
scritto:
Hi Michele,
On 10/01/2024 17:40, Michele Pescapè wrote:
> Hi,
>
> I'm attaching the configurations I'm currently using. I'll try
disabling
> KASLR.
[…]
/* high RAM */
{
.phys_start = 0x42100000,
.virt_start = 0x00101000,
.size = 0x8000000,
[…]
Alignment of your high ram is odd. Virt & Phys should have the same
alignment. Try to use .phys=.virt=0x42100000, I guess that should help.
Thanks,
Ralf
>
> Thank you for your help,
> Michele
>
> Il giorno mercoledì 10 gennaio 2024 alle 17:24:35 UTC+1 Ralf
Ramsauer ha
> scritto:
>
> Hi,
>
> On 09/01/2024 19:58, Michele Pescapè wrote:
> > Hi,
> >
> > Just an update, I was able to get the following output by
adding -c
> > "earlyprintk=ttyS0,115200" to the command line:
> >
> > Started cell "linux-1"
> >
> >
> > Invalid physical address chosen!
>
> could you please share your system and non-root cell configuration?
>
> It smells like you have an overlap with the communication region.
You
> can check that with tools/jailhouse-config-check.
>
> >
> >
> >
> >
> > Physical KASLR disabled: no suitable memory region!
>
> Disable KASLR in your kernel for the moment. You don't need it while
> ramping up your system.
>
> Thanks,
> Ralf
>
> >
> > [ 0.000000] Linux version 6.2.0-rc3 (root@mp-LINUX-DESKTOP)
> > (x86_64-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.11)
> 12.3.0, GNU
> > ld (GNU Binutils) 2.40) #2 SMP PREEMPT_DYNAMIC Sun Jan 7
> 18:35:23 CET 2024
> > [ 0.000000] Command line: earlyprintk=ttyS0,115200
> > [ 0.000000] KERNEL supported cpus:
> > [ 0.000000] Intel GenuineIntel
> > [ 0.000000] AMD AuthenticAMD
> > [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87
> floating
> > point registers'
> > [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE
> registers'
> > [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX
> registers'
> > [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]:
> 256
> > [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size
> is 832
> > bytes, using 'compacted' format.
> > [ 0.000000] signal: max sigframe size: 1776
> > [ 0.000000] BIOS-provided physical RAM map:
> > [ 0.000000] BIOS-e801: [mem
> 0x0000000000000000-0x000000000009efff] usable
> > [ 0.000000] printk: bootconsole [earlyser0] enabled
> > [ 0.000000] NX (Execute Disable) protection: active
> > [ 0.000000] extended physical RAM map:
> > [ 0.000000] reserve setup_data: [mem
> > 0x0000000000000000-0x0000000000001fff] usable
> > [ 0.000000] reserve setup_data: [mem
> > 0x0000000000002000-0x000000000000212b] usable
> > [ 0.000000] reserve setup_data: [mem
> > 0x000000000000212c-0x000000000009efff] usable
> > [ 0.000000] DMI not present or invalid.
> > [ 0.000000] Hypervisor detected: Jailhouse
> > [ 0.000000] tsc: Detected 3393.624 MHz processor
> > [ 0.000019] .text .data .bss are not marked as E820_TYPE_RAM!
> > [ 0.005751] last_pfn = 0x3630 max_arch_pfn = 0x400000000
> > [ 0.011039] x86/PAT: Configuration [0-7]: WB WC UC- UC WB
> WP UC- WT
> > Memory KASLR using RDRAND RDTSC...
> > [ 0.032374] Using GB pages for direct mapping
> > [ 0.036743] Kernel panic - not syncing: alloc_low_pages: can
not
> > alloc memory
> > [ 0.043712] CPU: 0 PID: 0 Comm: swapper Not tainted
6.2.0-rc3 #2
> > [ 0.049702] Call Trace:
> > [ 0.052134] <TASK>
> > [ 0.054219] dump_stack_lvl+0x3a/0x58
> > [ 0.057865] dump_stack+0x10/0x18
> > [ 0.061162] panic+0x15c/0x308
> > [ 0.064205] alloc_low_pages+0x7d/0x1c0
> > [ 0.068024] phys_pmd_init+0x1f2/0x2eb
> > [ 0.071758] phys_pud_init+0x114/0x2d5
> > [ 0.075493] phys_p4d_init+0x4c/0x269
> > [ 0.079135] ?
> __raw_callee_save___native_queued_spin_unlock+0x15/0x30
> > [ 0.085649] __kernel_physical_mapping_init+0x11e/0x29a
> > [ 0.090859] kernel_physical_mapping_init+0xf/0x20
> > [ 0.095631] init_memory_mapping+0x273/0x410
> > [ 0.099890] init_range_memory_mapping+0xec/0x150
> > [ 0.104574] init_mem_mapping+0x268/0x2e7
> > [ 0.108567] setup_arch+0x839/0xdcf
> > [ 0.112039] ? lockdown_lsm_init+0x21/0x2f
> > [ 0.116120] start_kernel+0x6e/0x997
> > [ 0.119680] x86_64_start_reservations+0x24/0x2c
> > [ 0.124281] x86_64_start_kernel+0xec/0xff
> > [ 0.128360] secondary_startup_64_no_verify+0xe5/0xeb
> > [ 0.133400] </TASK>
> > [ 0.135570] ---[ end Kernel panic - not syncing:
> alloc_low_pages: can
> > not alloc memory ]---
> > Closing cell "linux-1"
> > Page pool usage after cell destruction: mem 254/32211, remap
> 16392/131072
> > CPU 4 received SIPI, vector 96
> > CPU 5 received SIPI, vector 96
> >
> > This happens both with and without the initrd in the command,
> i'll try
> > to solve this and see what happens.
> >
> > By the way, is it possible that the new cell starts transmitting
> with a
> > baud rate different than the root cell? Without specifying the
> speed in
> > the command above the serial console stopped receiving anything
> until I
> > disabled and re-enabled the hypervisor. If not I may have
something
> > wrong on my end to solve
> >
> > Il giorno martedì 9 gennaio 2024 alle 17:37:48 UTC+1 Michele
> Pescapè ha
> > scritto:
> >
> > I stand corrected, the freeze happens when i do not set
> > JAILHOUSE_CELL_PASSIVE_COMMREG in the inmate
> >
> > Il giorno martedì 9 gennaio 2024 alle 17:26:57 UTC+1 Michele
Pescapè
> > ha scritto:
> >
> > Hi, thank you for replying.
> >
> > ttyS0 is indeed whitelisted in the inmate and the number of
> > pio_regions is set accordingly.
> >
> > As for the crash after disable/shutdown, I think it was related
> > to the PCI device, after commenting that and the corresponding
> > regions out I no longer get that freeze. CONFIG_JAILHOUSE_GUEST
and
> > CONFIG_JAILHOUSE_DBCON were already set in the kernel
> > configuration I'm using for the non root linux; still, I
> > recompiled it just in case.
> >
> > Michele
> >
> > Il giorno martedì 9 gennaio 2024 alle 14:14:33 UTC+1 Ralf
> > Ramsauer ha scritto:
> >
> > Hi,
> >
> > On 08/01/2024 23:56, Michele Pescapè wrote:
> > > Hi all,
> > >
> > > I have an AMD Ryzen 2600 on a b450 motherboard. My goal
> > is to run two
> > > non root linux cells, however, as of right now, I'm not
> > able to start
> > > any non root linux cells.
> > > First of all I don't receive an output from the cell (I
> > do get output
> > > from the apic-demo though), therefore I don't really know
> > if they're
> > > even crashing or not; this is the output I get from the
> > hypervisor after
> > > enabling the root cell and issuing the following command:
> > >
> > > sudo jailhouse cell linux configs/x86/linux_guest1.cell
> > > ../buildroot-2023.11/output/images/bzImage -i
> > > ../buildroot-2023.11/output/images/rootfs.cpio
> >
> > for the first few tries, simply don't load a ramdisk. Try to
> > get the
> > kernel booting. It will crash with "cannot mount rootfs". If
> > we get so
> > far, then you can continue specifying the ramdisk. But
> > first, we have to
> > get that far.
> >
> > >
> > > Adding PCI device ff:1f.7 to cell "linux-x86-demo"
> > > Created cell "linux-x86-demo"
> > > Page pool usage after cell creation: mem 336/32211, remap
> > 16392/131072
> > > Cell "linux-x86-demo" can be loaded
> > > CPU 4 received SIPI, vector 100
> > > CPU 3 received SIPI, vector 100
> > > Started cell "linux-x86-demo"
> > >
> > > After this the cell appears to be running in cell list,
> > however I can't
> > > seem to get any output to confirm that (I even set
> > > JAILHOUSE_CELL_VIRTUAL_CONSOLE_ACTIVE, but nothing
> > appears on the vrtual
> > > console accessed via "jailhouse console -f"). I also
> > tried adding -c
> > > "console=ttyS0,115200".
> >
> > ttyS0, i.e., x86 Port 0x3f8 is whitelisted in the cell
> > configuration?
> > Actually, your cmdline is correct.
> >
> > > At this point if I try to disable/shutdown the cell, the
> > system freezes
> > > requiring a hard reset.
> >
> > Okay, is the guest Linux compiled with Jailhouse guest
> > support? Please
> > ensure that non-root Linux is compiled with
> > CONFIG_JAILHOUSE_GUEST and
> > CONFIG_JAILHOUSE_DBCON. The first one is mandatory for x86
> > Linux guests.
> >
> > >
> > > Also, there seems to be a mismatch between the PCI bdf
> > value in the log
> > > above and the one in the cell's configuration. In both
> > root and non root
> > > cells i set .bdf = (0x0f << 3) for the IVSHMEM net device
> > and the Root
> > > cell correctly reports that ("Adding virtual PCI device
> > 00:0f.0 to cell
> > > "RootCell"").
> >
> > First things first. Let's try to get Linux kicked off before
> > diving into
> > PCI. Comment it out for the moment, let's do that later.
> >
> > Ralf
> >
> > >
> > > If I understood correctly, once the cell works properly I
> > won't need to
> > > set the console anymore as I can assign an ip (-c "ip
> > x.x.x.x" in the
> > > cell linux command) and ssh to the cell, right?
> > >
> > > I attach the root and non root cells' configurations in
> > case they're
> > > needed. I can provide both the root and non root cell's
> > kernel .conf and
> > > the buildroot configuration I used if needed.
> > >
> > > Thank you for your time,
> > > Michele
> > >
> > > --
> > > You received this message because you are subscribed to
> > the Google
> > > Groups "Jailhouse" group.
> > > To unsubscribe from this group and stop receiving emails
> > from it, send
> > > an email to jailhouse-de...@googlegroups.com
> > > <mailto:jailhouse-de...@googlegroups.com>.
> > > To view this discussion on the web visit
> > >
> >
>
https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com>
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com>>
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com>
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com>>>
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer>
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer>>
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer>
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer>>>>.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Jailhouse" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to jailhouse-de...@googlegroups.com
> > <mailto:jailhouse-de...@googlegroups.com>.
> > To view this discussion on the web visit
> >
>
https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com
<https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com>
<https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com
<https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com>>
<https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com?utm_medium=email&utm_source=footer>
<https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jailhouse" group.
> To unsubscribe from this group and stop receiving emails from it,
send
> an email to jailhouse-de...@googlegroups.com
> <mailto:jailhouse-de...@googlegroups.com>.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/jailhouse-dev/9a021e3f-caca-4ebe-9351-be2c3e999446n%40googlegroups.com
<https://groups.google.com/d/msgid/jailhouse-dev/9a021e3f-caca-4ebe-9351-be2c3e999446n%40googlegroups.com>
<https://groups.google.com/d/msgid/jailhouse-dev/9a021e3f-caca-4ebe-9351-be2c3e999446n%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/jailhouse-dev/9a021e3f-caca-4ebe-9351-be2c3e999446n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
--
You received this message because you are subscribed to the Google
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to jailhouse-dev+unsubscr...@googlegroups.com
<mailto:jailhouse-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/23a9c30e-5507-449e-9589-697904becac4n%40googlegroups.com <https://groups.google.com/d/msgid/jailhouse-dev/23a9c30e-5507-449e-9589-697904becac4n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/81df336d-a7bb-49e2-8762-a1294aef67e9%40oth-regensburg.de.
