Re: PVH boot with qemu
On Mon, 11 Dec 2023, Emile `iMil' Heitor wrote: We still need to check if we didn't break anything on Xen side and test Firecracker. FYI qemu-system-x86_64 also works with the "microvm" machine type. I am able to boot this patched NetBSD kernel using Colin Percival's PVH-enabled Firecracker https://github.com/firecracker-microvm/firecracker/pull/3155 Nevertheless, Firecracker, as the microvm qemu machine type, uses vioblk for block devices, which we don't have. Yet. Emile `iMil' Heitor | https://imil.net
Re: __futex(2): use outside Linux compat
> On Dec 11, 2023, at 5:00 AM, Robert Swindells wrote: > >> Is it OK to use for NetBSD "native" code since it is not "advertised" >> by a man page? > > No. Heavy asterisk here. Thierry, let’s chat off-list. -- thorpej
Re: __futex(2): use outside Linux compat
On Mon, Dec 11, 2023 at 01:00:38PM +, Robert Swindells wrote:
>
> tlaro...@kergis.com wrote:
> > In Mesa code implementations for futex_wake() and futex_wait() are
> > provided for Linux, Windows, FreeBSD and OpenBSD.
> >
> > There is a __futex(2) syscall in NetBSD, used only for now, if I'm not
> > mistaken, to implement Linux compat.
>
> The Linux emulation of futexes in NetBSD does not work correctly.
>
> > Is it OK to use for NetBSD "native" code since it is not "advertised"
> > by a man page?
>
> No.
OK, thanks for the precisions.
To state the problem: NetBSD userland is probably the sole user of the
non futex code in Mesa. Hence, since userland doesn't follow the same
code path as the same apps on other OSes, and since this code
(!UTIL_FUTEX_SUPPORTED) is a second rate citizen considering that the
main development (Linux) is taking another path, it could be that
the apps (the various Mesa libs components) are exercising bugs in
this part, the "tearing" or "threaded" (incorrect lines in a window)
that can be observed on NetBSD in certain circumstances being caused by
userlevel concurrent accesses, and not by kernel cache problems (there
have been reports that these defects are decreasing under heavy load
and this is perhaps only because under heavy loads there are less
threads concurrently running for the X clients, and they have no
occasion to trash shared zones that should be, normally, protected by
futexes).
So 3 options:
1) To fix the futex support on NetBSD ("ideal" solution but
quite involved, at leas for me);
2) Debug the non futex code in Mesa (meaning only finding if
the problems seen can come from there);
3) Let it be for now...
I will probably opt for 3) since I wanted to debug Mesa for other more
disastrous infelicities (crashes with xine(1) or vlc(1)---and probably
others since this comes from Mesa libs and probably not from the way
the API is used in the clients).
--
Thierry Laronde
http://www.kergis.com/
http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
Re: __futex(2): use outside Linux compat
tlaro...@kergis.com wrote: > In Mesa code implementations for futex_wake() and futex_wait() are > provided for Linux, Windows, FreeBSD and OpenBSD. > > There is a __futex(2) syscall in NetBSD, used only for now, if I'm not > mistaken, to implement Linux compat. The Linux emulation of futexes in NetBSD does not work correctly. > Is it OK to use for NetBSD "native" code since it is not "advertised" > by a man page? No.
__futex(2): use outside Linux compat
In Mesa code implementations for futex_wake() and futex_wait() are provided for Linux, Windows, FreeBSD and OpenBSD. There is a __futex(2) syscall in NetBSD, used only for now, if I'm not mistaken, to implement Linux compat. Is it OK to use for NetBSD "native" code since it is not "advertised" by a man page? -- Thierry Laronde http://www.kergis.com/ http://kertex.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
Re: PVH boot with qemu
On Mon, 11 Dec 2023, Manuel Bouyer wrote: Yes, right now GENERIC can be used on bare-metal, PVHVM and XENPVH. It would be good to have GENERIC working on GENPVH too. Fair enough, I'll switch to this path then, thanks for the advice. Emile `iMil' Heitor | https://imil.net
Re: PVH boot with qemu
On Mon, Dec 11, 2023 at 10:22:18AM +0100, Emile `iMil' Heitor wrote: > > Hi Manuel, > > On Mon, 11 Dec 2023, Manuel Bouyer wrote: > > > #ifndef GENPVH > > /* get a page for HYPERVISOR_shared_info */ > > addl$PAGE_SIZE, %ebx > > addl$PGOFSET,%ebx > > andl$~PGOFSET,%ebx > > movl$RELOC(HYPERVISOR_shared_info_pa),%ebp > > movl%ebx,(%ebp) > > movl$0,4(%ebp) > > #endif > > > > How can this work on Xen when GENPVH is defined ? > > Shouldn't this be made conditional on vm_guest == VM_GUEST_XENPVH ? > > Well the point is that you don't define GENPVH when using Xen, PVH using > qemu and friends don't need HYPERVISOR_shared_info neither any of the > hypercall portion of the code. A big chunk of Xen related code is > ifndef'ed to GENPVH in hypervisor.c; And I was planning on isolating GENPVH > so there's as little ifdef's as possible. > > Or would you prefer the same kernel to be able to boot in both XENPVH and > GENPVH modes? I am focusing on making the resulting kernel smaller but this > could be done also. Yes, right now GENERIC can be used on bare-metal, PVHVM and XENPVH. It would be good to have GENERIC working on GENPVH too. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: PVH boot with qemu
On Mon, Dec 11, 2023 at 10:22:18AM +0100, Emile `iMil' Heitor wrote: > Or would you prefer the same kernel to be able to boot in both XENPVH and > GENPVH modes? I am focusing on making the resulting kernel smaller but this > could be done also. Can you use a separate entry point and optimize the kernel size #if defined(GENPVH) && !defined(XEN) or something like that? Martin
Re: PVH boot with qemu
Hi Manuel, On Mon, 11 Dec 2023, Manuel Bouyer wrote: #ifndef GENPVH /* get a page for HYPERVISOR_shared_info */ addl$PAGE_SIZE, %ebx addl$PGOFSET,%ebx andl$~PGOFSET,%ebx movl$RELOC(HYPERVISOR_shared_info_pa),%ebp movl%ebx,(%ebp) movl$0,4(%ebp) #endif How can this work on Xen when GENPVH is defined ? Shouldn't this be made conditional on vm_guest == VM_GUEST_XENPVH ? Well the point is that you don't define GENPVH when using Xen, PVH using qemu and friends don't need HYPERVISOR_shared_info neither any of the hypercall portion of the code. A big chunk of Xen related code is ifndef'ed to GENPVH in hypervisor.c; And I was planning on isolating GENPVH so there's as little ifdef's as possible. Or would you prefer the same kernel to be able to boot in both XENPVH and GENPVH modes? I am focusing on making the resulting kernel smaller but this could be done also. Emile `iMil' Heitor | https://imil.net
Re: PVH boot with qemu
On Mon, Dec 11, 2023 at 08:26:01AM +0100, Emile `iMil' Heitor wrote: > > Here is a clean(er) patch > https://github.com/NetBSD/src/compare/trunk...NetBSDfr:NetBSD-src:GENPVH > > Rationale > > Like previously explained, locore.S expects start_info being passed by the > calling hypervisor on %ebx to be located at the end of the symbol table. > Qemu and Firecracker don't follow this rule which is not part of the > official Xen ABI https://xenbits.xen.org/docs/unstable/misc/pvh.html > > What our patch first does is make memory mapping loops happy by copying > the start_info structure where it's expected. > After that, memory locations and boot parameters are correctly found and > boot can proceed. > Of course, the hypervisor not being Xen, a lot of Xen-related code is > useless, hence the new VM_GUEST_GENPVH (for Generic PVH) vm_guest type, > as first suggested by Manuel, and a new kernel option, GENPVH. > I kept the Xen code structure as it was and changed very little code, only > some || vm_guest == VM_GUEST_GENPVH and a couple #ifndef GENPVH. > > In order to build a Generic PVH kernel, the following options are needed > > #Xen PV support for PVH and HVM guests > options XENPVHVM > options XEN > # Generic PVH support (qemu, firecracker...) > options GENPVH > > I've added > https://github.com/NetBSDfr/NetBSD-src/blob/GENPVH/sys/arch/amd64/conf/MICROVM > as an example config file. > I'll probably end up ditching XENPVHVM and XEN but there's still quite > some work in there. > > We still need to check if we didn't break anything on Xen side and test > Firecracker. FYI qemu-system-x86_64 also works with the "microvm" > machine type. > > Feedback very welcome. Hello, I don't understand this part: #ifndef GENPVH /* get a page for HYPERVISOR_shared_info */ addl$PAGE_SIZE, %ebx addl$PGOFSET,%ebx andl$~PGOFSET,%ebx movl$RELOC(HYPERVISOR_shared_info_pa),%ebp movl%ebx,(%ebp) movl$0,4(%ebp) #endif How can this work on Xen when GENPVH is defined ? Shouldn't this be made conditional on vm_guest == VM_GUEST_XENPVH ? -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
