Hi,
On 6/10/19 7:45 PM, Wayne wrote:
> Thank you for your responses Ralf and Jan.
>
> Yes, I have successfully gotten both the apic-demo and tiny-demo cells
> to work on my system before attempting to load the non-root linux. I
> can see serial output from them.
>
> However, if I try to share the UART between root and inmate for them I
> run into a crash on the hypervisor if the root linux attempts to write
> to the UART (after creating/starting a demo). Does the root linux lose
> access once an inmate is created/started? Or am I missing something
> from my System config or tiny-demo config that allows them to share? I
> attached the crash in hypervisor_output.txt.
>
> Also, I noticed the tiny-demo output on the UART gets mirrored on the
Yes, according to your config, JAILHOUSE_CELL_VIRTUAL_CONSOLE_ACTIVE is
set. This means that the guest will use the virtual console. The virtual
console uses the hypervisor output which depends whatever is set in your
system configuration. In your case the hypervisor uses 3f8.
And additionally, the inmate also a valid .console set. Again, 3f8. This
means, it will use the console *and* the hypervisor debug hypercall. In
your case, both outputs are routed to the same device which explains the
mirroring.
> Jailhouse virtual console. Does that mean that its writing to the
> serial through the hypervisor and not directly accessing ttyS0 itself?
Exactly.
> If so, is there a way to create the cell to write ttyS0 directly as a
> test? To see if it exhibits same behavior as the non-root linux.
Sure, simply align the .address field of .console, and allow access to
that port via .pio_bitmap.
>
> FYI, I create the tiny-demo like this:
>
> jailhouse cell create configs/x86/tiny-demo.cell
> jailhouse cell load tiny-demo inmates/demos/x86/tiny-demo.cell
> jailhouse cell start tiny-demo
>
> Going back to my original non-root linux question, I don't see any
> output when connected over ttyS0. Its address is 0x3f8 ("serial 1") in
> the PIO, and that seems to work fine for the other simple demos, minus
> the hypervisor crash issue above when sharing. I know there is the
> other "serial 2" port but I haven't tried to use that one.
I see that port on our Dell server as well. It's present, but I don't
know where it's connected to.
Let me just summarise your issue:
You only have one serial line available, right? From the root-cell's
POV, it's ttyS0 on 3f8.
I would propose to share ttyS0 between the hypervisor and non-root
Linux. This means:
- Don't let the root-cell use ttyS0. Remove any console=ttyS0
parameters from you commandline. Ensure that noone else accesses
ttyS0 (e.g., getty@ttyS0 or other friends)
- Set 3f8 as debug_console in your master.c (seems you already have)
- Allow 3f8 access in the linux-demo.c (already set in the default
linux-demo
- Disallow 2f8 access in linux-demo.c:
- [ 0x2f8/8 ... 0x2ff/8] = 0, /* serial2 */
+ [ 0x2f8/8 ... 0x2ff/8] = -1, /* serial2 */
I don't know how Linux enumerates serial devices, but this ensures
that Linux won't see the unconnected serial line and map ttyS0 to
3f8
>
> non root linux launched with:
> /tools/jailhouse cell linux configs/x86/linux-x86.cell
> /boot/vmlinux-4.1.16-Guest -c "console=ttyS0,115200"
Then, this should actually work...
And if not, then console=jailhouse0 should work.
>
> I can try the 4.19 siemens kernel and "next" branch for jailhouse you
> suggested. Do you think that combination will resolve the missing
> serial for the non-root linux? Otherwise, did my kernel config, system
> config, and non-root linux cell configs look OK?
I didn't look at it, but we should at least - even if some flags are
still incorrect - see the "Uncompressing kernel..." thing.
Wait -- one thing you could try: Deactivate CONFIG_EFI and especially
CONFIG_EFI_STUBS. I'm not sure, but it could be that EFI_STUBS change
the header format that our bootloader patches.
>
> Thanks again for your help.
No problem. But it's not yet working. ;-)
Ralf
>
>
> On Fri, Jun 7, 2019 at 5:30 PM Ralf Ramsauer
> <ralf.ramsa...@oth-regensburg.de
> <mailto:ralf.ramsa...@oth-regensburg.de>> wrote:
>
> Hi Wayne,
>
> On 6/7/19 10:04 PM, Wayne wrote:
> > Hi Ralf,
> >
> > Thank you for your responses. I have attached the requested config
> > files. I am now just trying to use the same bzImage for the root and
> > the non-root linux node.
>
> One question that you didn't answer: do apic-demo or tiny-demo work?
> They need to to have correct .console parameter set, so ensure that
> parameters are set if you run those demos in your linux-demo cell. Just
> compare it to tiny-demo.c or apic-demo.c
>
> >
> > Yes, I am using vanilla 4.16 because the documentation indicated
> that it
> > is the first mainline release to officially support Jailhouse Guest
> > images. Do you suggest that I use something else?
>
> As Jan already wrote: Try to use Siemens' 4.19 branch.
>
> >
> > Right now i'm just trying to get some serial output from the non-root
> > Linux. Hopefully, I have that configured correctly. Ideally, I would
> > like to share the same serial console for the Root node and non-root
> > node eventually if possible. I have been OK with using the virtual
> > console (jailhouse console -f) for the Hypervisor output.
>
> In addition to Jan's comment:
>
> Currently you only have access to the jailhouse console via 'jailhouse
> console' from your root cell. Imagine your root cell crashes for some
> reason. You will never see the fault reason, which makes things hard to
> debug.
>
> I would suggest to configure the hypervisor to use the serial console.
> You can share the device with the non-root Linux, that's no problem.
>
> In this case, the non-root kernel's output will always be printed to the
> serial console. Directly, if you choose console=ttySx, and indirectly
> via the hypervisor if you choose console=jailhouse.
>
> BTW: According to your config, your system is a PowerEdge, and the
> non-root cell gets both, 0x2f8 and 0x3f8. Are you sure that ttyS1 is the
> correct console that you are connected to?
>
> Just mentioning this as we have a similar machine here, and, afair, both
> platform serial device are 'usable', but one ends in the nirvana while
> it is accessible.
>
> Ah, and one last thing: try to switch to the current next branch for
> jailhouse. Jan just integrated a few patches from me that might also be
> relevant for your machine.
>
> HTH
> Ralf
>
> >
> > Wayne
> >
> > On Fri, Jun 7, 2019 at 9:10 AM Ralf Ramsauer
> > <ralf.ramsa...@oth-regensburg.de
> <mailto:ralf.ramsa...@oth-regensburg.de>
> > <mailto:ralf.ramsa...@oth-regensburg.de
> <mailto:ralf.ramsa...@oth-regensburg.de>>> wrote:
> >
> > Hi,
> >
> > On 6/7/19 2:28 PM, Wayne wrote:
> > > Hello,
> > >
> > > I am new to Linux development and Jailhouse. I have
> successfully
> > booted
> > > the Jailhouse Hypervisor and root cell on a bare metal X86 Linux
> > system
> > > (No QEMU). I am now trying to load a non-root Linux cell
> and I have a
> > > few questions. Jailhouse accepts and starts my non-root
> linux cell
> > > configuration and I see it as "running" through the
> "jailhouse cell
> > > list" command. However, I don't see any serial output from the
> > > "non-root linux" cell booting up. I’m not sure what the
> non-root node
> > > is doing at this point.
> >
> > Ok, first of all, can you see any output from Jailhouse's demo
> inmates
> > (e.g., tiny-demo or apic-demo)?
> >
> > >
> > > My root node is a 4.16 kernel configured this
> way:
> > > 1. CONFIG_JAILHOUSE_GUEST is
> not set
> > > 2. CONFIG_SERIO=y
> > > 3.
> CONFIG_SERIAL_8250_RUNTIME_UARTS=4
> > >
> > > My non-root node is a 4.16 kernel configured
> this way:
> > > 1. CONFIG_JAILHOUSE_GUEST=y
> > > 2. CONFIG_SERIO=m (can't seem
> to disable
> > > completely in my config for 4.16)
> > > 3.
> CONFIG_SERIAL_8250_RUNTIME_UARTS=1
> >
> > Could you please provide a full .config? What branch are you
> exactly
> > using? Vanilla 4.16?
> >
> > Please attach your system config and your non-root linux
> config as well.
> >
> > You may also want to try https://github.com/siemens/linux . See
> > jailhouse-enabling branches for some releases.
> >
> > >
> > > In general, do the kernel config settings have to match between
> > the root
> > > node and non-linux or is the above fine?
> >
> > No, they do not have to be the same, but they can. Still, an
> analysis
> > requires your .config.
> >
> > >
> > > The vmlinux-4.1.16-Guest bzImage is approx 7MB, and the
> inmate node is
> > > allocated ~75MB of RAM.
> > >
> > > I have a single UART, so I have configured the root cell
> system config
> > > to output to the virtual hypervisor console:
> >
> > Wait... You configured your root-cell to use the hypervisor debug
> > console? How? It's only available once the hypervisor is
> enabled. How
> > should this work?
> >
> > But let's have a look at your configuration first.
> >
> > So you want to:
> > - Have the UART (0x3f8) available in the non-root cell
> > - Use 0x3f8 as hypervisor debug console as well
> > - No console for root-cell
> >
> > Did I get this right?
> >
> > Ralf
> >
> > >
> > > .flags = JAILHOUSE_SYS_VIRTUAL_DEBUG_CONSOLE,
> > >
> > > .debug_console = {
> > > .type = JAILHOUSE_CON_TYPE_NONE,
> > > },
> > >
> > > and I have configured the non-root linux cell to output to
> the UART:
> > >
> > > (Added serial 0x3f8 to pio bitmap for non-root linux) and
> started the
> > > node with this:
> > >
> > > ./tools/jailhouse cell linux configs/x86/linux-x86.cell
> > > /boot/vmlinux-4.1.16-Guest -c "console=ttyS0,115200"
> > > (Note I also tried "console=jailhouse" in the command above,
> and that
> > > produces the same result)
> > >
> > > I then see the following on my hypervisor console
> (./tools/jailhouse
> > > console -f):
> > >
> > > Created cell "linux-x86-demo"
> > > ...
> > > Cell "linux-x86-demo" can be loaded
> > > Started cell "linux-x86-demo"
> > >
> > > After a little while I do get a parked CPU error on the root
> node,
> > looks
> > > like its trying to do something with the UART as well:
> > > FATAL: Invalid PIO read, port: 3fe size: 1
> > >
> > > I would expect something to pop out on the UART from the
> non-root
> > linux
> > > node first. Note that root node has serial 0x3f8 disabled
> in its pio
> > > bitmap.
> > >
> > > I verifed that the UART is functioning by allowing the
> hypervisor to
> > > print to it and also performed an echo test over ttyS0.
> > >
> > > I have tried several configurations of kernel.....including your
> > current
> > > "queues/jailhouse" branch head kernel for the non-root node,
> along
> > with
> > > the kernel config for 4.7 posted in this thread below (but I
> get same
> > > result as above when I start it, no kernel output):
> > >
> > >
> >
>
> "https://groups.google.com/forum/#!searchin/jailhouse-dev/Re$3A$20Failed$20to$20boot$20jailhouse%7Csort:relevance/jailhouse-dev/M7UO89XFIk0/Qi40DDuMBAAJ";.
> > >
> > > Any information you can provide to me will be helpful. I'm
> not sure
> > > what might be going wrong here.
> > >
> > > Thanks,
> > > Wayne
> > >
> > > --
> > > You received this message because you are subscribed to the
> Google
> > > Groups "Jailhouse" group.
> > > To unsubscribe from this group and stop receiving emails
> from it, send
> > > an email to jailhouse-dev+unsubscr...@googlegroups.com
> <mailto:jailhouse-dev%2bunsubscr...@googlegroups.com>
> > <mailto:jailhouse-dev%2bunsubscr...@googlegroups.com
> <mailto:jailhouse-dev%252bunsubscr...@googlegroups.com>>
> > > <mailto:jailhouse-dev+unsubscr...@googlegroups.com
> <mailto:jailhouse-dev%2bunsubscr...@googlegroups.com>
> > <mailto:jailhouse-dev%2bunsubscr...@googlegroups.com
> <mailto:jailhouse-dev%252bunsubscr...@googlegroups.com>>>.
> > > To view this discussion on the web visit
> > >
> >
>
> https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc2iKk1J6%2B0huh5__dS4HyujXzV9r%2BLbKLzuVZ4K3Bt5eA%40mail.gmail.com
> > >
> >
>
> <https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc2iKk1J6%2B0huh5__dS4HyujXzV9r%2BLbKLzuVZ4K3Bt5eA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> > > For more options, visit https://groups.google.com/d/optout.
> >
>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/f0936088-a32d-3da7-a9ba-d552ede5052e%40oth-regensburg.de.
For more options, visit https://groups.google.com/d/optout.
