On 8/14/20 7:20 PM, Jan Kiszka wrote: > On 14.08.20 19:06, Ralf Ramsauer wrote: >> >> >> On 8/14/20 5:47 PM, Jan Kiszka wrote: >>> On 14.08.20 17:06, Ralf Ramsauer wrote: >>>> (+Cc Wolfgang, we recently had discussion on the boot process of >>>> Jailhouse) >>>> >>>> On 8/14/20 11:43 AM, Jan Kiszka wrote: >>>>> On 14.08.20 11:08, Peng Fan wrote: >>>>>>> Subject: Re: [PATCH 06/10] Add libbaremetal >>>>>>> >>>>>>> On 07.08.20 05:06, peng....@nxp.com wrote: >>>>>>>> From: Peng Fan <peng....@nxp.com> >>>>>>>> >>>>>>>> Add libbaremetal for sharing code between inmates and jailhouse >>>>>>>> baremetal loader. >>>>>>>> >>>>>>>> The Makefile code is copied from inmates, currently only >>>>>>>> string.c is >>>>>>>> moved from inmates to libbaremetal. In future, we might need to >>>>>>>> share >>>>>>>> uart/mmu and etc. >>>>>>> >>>>>>> Might quickly become confusing to have two libs. What prevents >>>>>>> renaming >>>>>>> inmates/lib completely into libbaremetal? Sure, there are some >>>>>>> jailhouse >>>>>>> specifics in inmates/lib, but those could likely be put in some >>>>>>> corner. >>>>>> >>>>>> How about rename inmates to baremetal? >>>>>> And add bootloader stuff under baremetal? >>>>> >>>>> We could do >>>>> >>>>> baremetal >>>> >>>> What we provide is a minimalist runtime environment, which is not just >>>> limited to baremetal applications (depends on the definition of >>>> baremetal, though). But… >>>> >>>>> ├── demo-inmates >>>>> ├── lib >>>>> ├── tests >>>>> └── tools >>>>> >>>>> and put the jailhouse loader under tools. Inmates is a "brand" in >>>>> Jailhouse context, so it should remain in some form. But it's true, >>>>> some >>>> >>>> … yep, the branding is the point why we should keep calling them >>>> 'inmates'. >>>> >>>>> of the existing demo inmates can already run without jailhouse. >>>> >>>> And that's the second point - we already have some kind of support for >>>> this kind of use case. However, I agree that I wouldn't expect the >>>> preloader inside inmates/. >>>> >>>>> >>>>> Ralf, what do you think? >>>> >>>> Peng, I just read the code of the loader and I try to understand what >>>> how it's exactly working. Nice work! Do I see that correctly: You >>>> basically "imitate" Linux in a way that you manually do the early setup >>>> (some register setup, GIC init, kick off all CPUs, hv stubs, ...). >>>> >>>> IOW, you bring the system to the same state, where the Linux driver >>>> would hand over Jailhouse. You activate Jailhouse, we then return to >>>> the >>>> loader which got lifted to the first cell, the root cell. Now you use >>>> the loader to partition the system, i.e., to create cells. Therefore, >>>> you use the hypercall interface, just like Linux would typically do it >>>> for us. Finally, there's the hand over to Linux. >>>> >>>> But do we really need a separate loader? Can't we just place the >>>> loader-related stuff at, say, the end of jailhouse.bin? >>>> >>>> To be more concrete, i think it should be possible to have the full >>>> preloader-thing located under hypervisor/arch/arm64/. It should also be >>>> possible to link everything that is required into jailhouse.bin - so we >>>> would get one universal binary instead of a chain of dependent >>>> binaries. [1] >>> >>> Interesting idea. >>> >>>> >>>> Pro: You can make early use of the UART in your preloader without >>>> having >>>> the need to, for example, duplicate drivers. That's quite useful for >>>> debugging if something goes wrong very early. Secondly, you can also >>>> piggyback on existing gic routines, no need to reimplement >>>> phys_processor_id, … >>>> >>>> The next thing after early boot would be the handover to jailhouse. >>>> Before the handover, we set a flag that indicates that we do an early >>>> boot of Jailhouse. We can use this flag to do the full partitioning of >>>> the system before we return to the guests in one single step. IOW, we >>>> never return to the preloader once the handover happened. We can do >>>> that, as u-boot already placed our guests and we just need to start >>>> them >>>> (as we would start regular cells). >>>> >>>> The amount of additional logic in the actual hypervisor core should be >>>> reasonably low. >>>> >>>> Did you already consider that strategy? Jan, what do you think? >>> >>> As we are already in visionary phase... There is another use case to >>> consider: >>> >>> I'd like to have a Jailhouse variant that comes without a root cell. >>> That means, we will need a booting procedure that creates all cells, >>> loads and starts them - and then drops all the logic that could >>> configure or destroy them (or never includes it in the first place - >>> build-time setting). This should reduce the Jailhouse runtime code to >>> much less than 10K LoC. >> >> Ok, that's indeed an awesome idea. >> >>> >>> Peng's approach is useful in case you do want to keep the flexibility to >>> reconfigure the system from a root cell, just like you can do when >> >> To be honest, I consider the "dynamic" reconfigurability of Jailhouse as >> a development and testing utility rather than something that would ever >> be useful for operative usage. This includes, of course, jailhouse >> disabling. > > Reconfiguration, yes, but reloading can be a topic in certain scenarios. > The current config locking mechanisms allows to keep this door for the > root cell open, even while there is a critical cell running. > >> >> What would be great would be a single-shot 'jailhouse freeze' (or, ha, >> 'jailhouse detention') call, that throws away the whole hypercall >> interface. Having such a thing, we an drop everything that is somehow >> related to cell management: create, start, stop, destroy, stats, >> disable, enable, ... >> >> And that would in fact be no big deal: Annotate all affected routines, >> replace hyp vectors with stubs, drop the section, done. Every code that >> has no other users than from a hypercall can be dropped. Or do I miss >> something? > > I haven't thought this through in details yet, but I would be surprised > if there weren't at least some smaller challenges in implementing that > cleanly. Still, it's mostly about disabling code. > >> >>> booting via Linux. The idea of adding a preloader is minimal invasive to >>> the existing setup. My root-less mode will be maximal invasive, though. >> >> Why do you think that it would be maximal invasive? > > As it changes the current hypervisor code for sure. Just loading > Jailhouse via a different thing than Linux does not bring many changes, > at least to the core. See this series. > >> >> Oh btw: who would receive the freed memory? Would it remain in the >> hypervisor or be assigned back to a cell? Though that should probably >> only be a few pages. > > What freed memory? When things are only created, nothing will be freed > anymore. There will also be no jailhouse disable.
There will, e.g., be a cell_create routine, that will be called once per cell. After the creation of those cells, drop it, we won't need it again. If we have a universal binary, there will be a jailhouse disable will be present in the beginning. Once we decide freeze the configuration, drop it. If we make those things compile-time configurable, then it has potential to become a variant hell. > >> >>> If root-less would be sufficient for cases you do not want Linux-based >>> boot, we can skip this approach and head for root-less full steam. >> >> root-less means that the detention call (I like that) comes after cell >> creation, but before cell start. .oO(we can even drop the cell_start >> code, if we find a smart strategy to drop code right before vmreturn) >> > > I would rather envision some alternative configuration that contains all > cell configs, not just the root cell (which would become a non-root cell > as well). Loading would have to happen into the target memory regions > before jailhouse enable, and the enabling would also imply starting all > cells. > >>> >>> If there are use cases for all three variant, we can try to look for >>> common pieces in the two Linux-free options. My root-less mode would >>> also need a single-step "create and start all cells", e.g. And your idea >>> to include the bootstrap logic into an init section of jailhouse.bin is >>> also attractive for both. >> >> Yes, I see the overlaps. By the way... Why don't we actually do: >> >> $ cat jh.bin sys.cell c1.cell c2.cell > jh.image Even better: cat preloader.bin jh.bin sys.cell c1.cell c2.cell > jh.image Again, the preloader.bin is still a binary image that only contains the preloader section, but u-boot can directly jump into it. It is still developed under hypervisor/arch/foo and hence has full visibility of all symbols that are available in regular jailhouse code and can potentially use them. We just exploit the linker to export that section to a separate binary. If someone wants bare-metal boot mode, we just glue things together. >> >> for the bare metal case? > > That comes close to what I have in mind, config-wise. sys.cell would > still need a counter of cells that follow (simple to add to struct > jailhouse_system). Not necessarily. We can also define a zero-termination. Once the first byte of the JHCELL magic isn't present, we're done. But that's details. Ralf > > Jan > -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/9d551e5e-25fb-4c8f-66a4-934187b9650f%40oth-regensburg.de.
