Thanks for the quick reply. I haven't seen this failure when running under Qemu, but since I'm almost always running under Firecracker and it happens so infrequently under Firecracker, that's no surprise. I will try to reproduce under Qemu and hopefully collect more information.
On a related point, when this occurs (and I assume the behaviour would be the same with any kernel assert), the OS seems to be left in a hung state. What I'd like to ensure with my build is that if a fatal error occurs, the VM is powered off. Is there an option I can use to achieve this, or would it require code modification (perhaps injecting a call to poweroff() at an appropriate location)? On Monday, February 15, 2021 at 12:16:55 PM UTC Nadav Har'El wrote: > On Mon, Feb 15, 2021 at 1:51 PM David Smith <[email protected]> wrote: > >> I am running a Golang application on OSv under firecracker. Occasionally >> (it's proving difficult to reproduce reliably) I see an assertion failure >> early in the OSv startup sequence. The console output for my release build >> is as follows: >> >> OSv v0.55.0-168-g31a04239 >> 1 CPUs detected >> Firmware vendor: Unknown >> bsd: initializingAssertion failed: sched::preemptable() (arch/x64/mmu.cc: >> page_fault: 37) >> >> [backtrace] >> > > I wonder why the backtrace is empty. If you run with qemu, not > firecracker, I you can perhaps get a better backtrace with gdb - see > instructions in > https://github.com/cloudius-systems/osv/wiki/Debugging-OSv#debugging-osv-with-gdb. > > Does it fail in a similar way in qemu too, or just in firecracker? > > >> It appears to be hitting the assert well before my go application is >> launched, so my impression is that this isn't being caused by anything my >> application is doing, >> > > That's my impression too - bsd_init() finishes (and prints "done()") > before starting to run the application. > > >> though I haven't been able to prove that the same behaviour occurs with a >> base image (no apps included). >> > > This assert appears to happen in bsd_init() (although it's hard to know > without a backtrace) - and that runs with IRQs disabled so indeed should > not have had page faults and the fact we got a page fault might indicate a > bug. Hopefully the debugger or backtrace can explain why/where this page > fault is happening. > > > As you can see the backtrace is empty (perhaps that's the norm for a >> release build?), so there are no additional clues. If I can find a way to >> reproduce with a debug build and/or with a base image, I will update this >> thread. >> >> In most cases, my OSv image starts up without issue and in those cases, >> the console output around the same area looks as follows: >> >> OSv v0.55.0-168-g31a04239 >> 1 CPUs detected >> Firmware vendor: Unknown >> bsd: initializing - done >> VFS: mounting ramfs at / >> VFS: mounting devfs at /dev >> net: initializing - done >> >> I can see there are numerous previous discussions around the page fault >> when preemption is disabled assertion, but these seem to be tied to >> application behaviour. >> Is an assert during "bsd: Initializing" a known issue for which there is >> already a patch? >> > > Unfortunately, I don't remember this specific case was seen. We had > recently https://github.com/cloudius-systems/osv/issues/1123 where > bsd_init() hangs, though, I wonder if it's related. > > >> Otherwise, if there is additional info. that I can capture to help >> isolate the cause, let me know. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "OSv Development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/17c2a0f9-970f-4b8a-b2c4-18b3c678f027n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/osv-dev/17c2a0f9-970f-4b8a-b2c4-18b3c678f027n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/74d86949-792e-4bdd-885a-65aaf4819e21n%40googlegroups.com.
