On 14.08.20 19:47, Ralf Ramsauer wrote:
On 8/14/20 7:20 PM, Jan Kiszka wrote:
On 14.08.20 19:06, Ralf Ramsauer wrote:
On 8/14/20 5:47 PM, Jan Kiszka wrote:
On 14.08.20 17:06, Ralf Ramsauer wrote:
(+Cc Wolfgang, we recently had discussion on the boot process of
Jailhouse)
On 8/14/20 11:43 AM, Jan Kiszka wrote:
On 14.08.20 11:08, Peng Fan wrote:
Subject: Re: [PATCH 06/10] Add libbaremetal
On 07.08.20 05:06, peng....@nxp.com wrote:
From: Peng Fan <peng....@nxp.com>
Add libbaremetal for sharing code between inmates and jailhouse
baremetal loader.
The Makefile code is copied from inmates, currently only
string.c is
moved from inmates to libbaremetal. In future, we might need to
share
uart/mmu and etc.
Might quickly become confusing to have two libs. What prevents
renaming
inmates/lib completely into libbaremetal? Sure, there are some
jailhouse
specifics in inmates/lib, but those could likely be put in some
corner.
How about rename inmates to baremetal?
And add bootloader stuff under baremetal?
We could do
baremetal
What we provide is a minimalist runtime environment, which is not just
limited to baremetal applications (depends on the definition of
baremetal, though). But…
├── demo-inmates
├── lib
├── tests
└── tools
and put the jailhouse loader under tools. Inmates is a "brand" in
Jailhouse context, so it should remain in some form. But it's true,
some
… yep, the branding is the point why we should keep calling them
'inmates'.
of the existing demo inmates can already run without jailhouse.
And that's the second point - we already have some kind of support for
this kind of use case. However, I agree that I wouldn't expect the
preloader inside inmates/.
Ralf, what do you think?
Peng, I just read the code of the loader and I try to understand what
how it's exactly working. Nice work! Do I see that correctly: You
basically "imitate" Linux in a way that you manually do the early setup
(some register setup, GIC init, kick off all CPUs, hv stubs, ...).
IOW, you bring the system to the same state, where the Linux driver
would hand over Jailhouse. You activate Jailhouse, we then return to
the
loader which got lifted to the first cell, the root cell. Now you use
the loader to partition the system, i.e., to create cells. Therefore,
you use the hypercall interface, just like Linux would typically do it
for us. Finally, there's the hand over to Linux.
But do we really need a separate loader? Can't we just place the
loader-related stuff at, say, the end of jailhouse.bin?
To be more concrete, i think it should be possible to have the full
preloader-thing located under hypervisor/arch/arm64/. It should also be
possible to link everything that is required into jailhouse.bin - so we
would get one universal binary instead of a chain of dependent
binaries. [1]
Interesting idea.
Pro: You can make early use of the UART in your preloader without
having
the need to, for example, duplicate drivers. That's quite useful for
debugging if something goes wrong very early. Secondly, you can also
piggyback on existing gic routines, no need to reimplement
phys_processor_id, …
The next thing after early boot would be the handover to jailhouse.
Before the handover, we set a flag that indicates that we do an early
boot of Jailhouse. We can use this flag to do the full partitioning of
the system before we return to the guests in one single step. IOW, we
never return to the preloader once the handover happened. We can do
that, as u-boot already placed our guests and we just need to start
them
(as we would start regular cells).
The amount of additional logic in the actual hypervisor core should be
reasonably low.
Did you already consider that strategy? Jan, what do you think?
As we are already in visionary phase... There is another use case to
consider:
I'd like to have a Jailhouse variant that comes without a root cell.
That means, we will need a booting procedure that creates all cells,
loads and starts them - and then drops all the logic that could
configure or destroy them (or never includes it in the first place -
build-time setting). This should reduce the Jailhouse runtime code to
much less than 10K LoC.
Ok, that's indeed an awesome idea.
Peng's approach is useful in case you do want to keep the flexibility to
reconfigure the system from a root cell, just like you can do when
To be honest, I consider the "dynamic" reconfigurability of Jailhouse as
a development and testing utility rather than something that would ever
be useful for operative usage. This includes, of course, jailhouse
disabling.
Reconfiguration, yes, but reloading can be a topic in certain scenarios.
The current config locking mechanisms allows to keep this door for the
root cell open, even while there is a critical cell running.
What would be great would be a single-shot 'jailhouse freeze' (or, ha,
'jailhouse detention') call, that throws away the whole hypercall
interface. Having such a thing, we an drop everything that is somehow
related to cell management: create, start, stop, destroy, stats,
disable, enable, ...
And that would in fact be no big deal: Annotate all affected routines,
replace hyp vectors with stubs, drop the section, done. Every code that
has no other users than from a hypercall can be dropped. Or do I miss
something?
I haven't thought this through in details yet, but I would be surprised
if there weren't at least some smaller challenges in implementing that
cleanly. Still, it's mostly about disabling code.
booting via Linux. The idea of adding a preloader is minimal invasive to
the existing setup. My root-less mode will be maximal invasive, though.
Why do you think that it would be maximal invasive?
As it changes the current hypervisor code for sure. Just loading
Jailhouse via a different thing than Linux does not bring many changes,
at least to the core. See this series.
Oh btw: who would receive the freed memory? Would it remain in the
hypervisor or be assigned back to a cell? Though that should probably
only be a few pages.
What freed memory? When things are only created, nothing will be freed
anymore. There will also be no jailhouse disable.
There will, e.g., be a cell_create routine, that will be called once per
cell. After the creation of those cells, drop it, we won't need it again.
Right, but that's just init code then, like the stuff we already have in
hypervisor/setup.c. I thought you meant "real" memory.
If we have a universal binary, there will be a jailhouse disable will be
present in the beginning. Once we decide freeze the configuration, drop
it. If we make those things compile-time configurable, then it has
potential to become a variant hell.
Dropping significant code chunks from a root-less jailhouse.bin would be
a valuable reason for such a config variant. You do not have to argue
over things that are provably not there.
If root-less would be sufficient for cases you do not want Linux-based
boot, we can skip this approach and head for root-less full steam.
root-less means that the detention call (I like that) comes after cell
creation, but before cell start. .oO(we can even drop the cell_start
code, if we find a smart strategy to drop code right before vmreturn)
I would rather envision some alternative configuration that contains all
cell configs, not just the root cell (which would become a non-root cell
as well). Loading would have to happen into the target memory regions
before jailhouse enable, and the enabling would also imply starting all
cells.
If there are use cases for all three variant, we can try to look for
common pieces in the two Linux-free options. My root-less mode would
also need a single-step "create and start all cells", e.g. And your idea
to include the bootstrap logic into an init section of jailhouse.bin is
also attractive for both.
Yes, I see the overlaps. By the way... Why don't we actually do:
$ cat jh.bin sys.cell c1.cell c2.cell > jh.image
Even better:
cat preloader.bin jh.bin sys.cell c1.cell c2.cell > jh.image
Again, the preloader.bin is still a binary image that only contains the
preloader section, but u-boot can directly jump into it. It is still
developed under hypervisor/arch/foo and hence has full visibility of all
symbols that are available in regular jailhouse code and can potentially
use them. We just exploit the linker to export that section to a
separate binary.
Probably doable - will just blur our stats about what is hypervisor core
and what is boot loader.
If someone wants bare-metal boot mode, we just glue things together.
You will still need some U-Boot scripting to load all the cells,
provided that needs to happen early as well.
for the bare metal case?
That comes close to what I have in mind, config-wise. sys.cell would
still need a counter of cells that follow (simple to add to struct
jailhouse_system).
Not necessarily. We can also define a zero-termination. Once the first
byte of the JHCELL magic isn't present, we're done. But that's details.
Given
https://groups.google.com/d/msgid/jailhouse-dev/20200807030632.28259-8-peng.fan%40nxp.com,
we will have a spare field anyway. ;)
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/4f2c2e19-eba7-6020-5f1c-137f9e8c5efe%40siemens.com.
