Can you run the ROFS image you built? Also as I understand it NIX is a package manager but what Linux distribution are you using?
As far as ZFS goes could you enable ELF debugging - change this line: conf-debug_elf=0 To conf-debug_elf=1 In conf/base.mk, delete core/elf.o and force rebuild the kernel. I think you may also need to change the script upload_manifest.py to peeped ‘—verbose’ to the command line with cpiod.so It should show more info about elf loading. It may still be necessary to add extra printouts to capture which exact elf it is crashing on in arch_relocate_jump(). In worst case I would need a copy of your loader-stripped.elf and possibly all the other files like cpiod.so, zfs.so that go into the bootfs part of the image. Regards, Waldek On Sat, Dec 5, 2020 at 19:31 Matthew Kenigsberg <[email protected]> wrote: > After forcing it to use the right path for libz.so.1, it's working with > rofs, but still having the same issue when using zfs, even after I correct > the path for libz. > > On Saturday, December 5, 2020 at 5:18:37 PM UTC-7 Matthew Kenigsberg wrote: > >> gcc version 9.3.0 (GCC) >> QEMU emulator version 5.1.0 >> >> Running with fs=rofs I get the error: >> Traceback (most recent call last): >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 369, in <module> >> main() >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 366, in main >> gen_image(outfile, manifest) >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 269, in gen_image >> system_structure_block, bytes_written = write_fs(fp, manifest) >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 246, in write_fs >> count, directory_entries_index = write_dir(fp, manifest.get(''), '', >> manifest) >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 207, in write_dir >> count, directory_entries_index = write_dir(fp, val, dirpath + '/' + >> entry, manifest) >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 207, in write_dir >> count, directory_entries_index = write_dir(fp, val, dirpath + '/' + >> entry, manifest) >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 222, in write_dir >> inode.count = write_file(fp, val) >> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 164, in >> write_file >> with open(path, 'rb') as f: >> FileNotFoundError: [Errno 2] No such file or directory: 'libz.so.1' >> >> I think that's from this line in usr.manifest? >> /usr/lib/libz.so.1: libz.so.1 >> >> Don't have zlib in the manifest without fs=rofs, and I think zpool uses >> it? >> >> Looking into it... >> On Saturday, December 5, 2020 at 4:36:20 PM UTC-7 [email protected] >> wrote: >> >>> I can not reproduce it on Ubuntu 20.20 neither Fedora 33. Here is the >>> code fragment where it happens: >>> >>> 169 bool object::arch_relocate_jump_slot(symbol_module& sym, void *addr, >>> Elf64_Sxword addend) >>> >>> 170 { >>> >>> 171 if (sym.symbol) { >>> >>> 172 *static_cast<void**>(addr) = sym.relocated_addr(); >>> >>> 173 return true; >>> >>> 174 } else { >>> >>> 175 return false; >>> >>> 176 } >>> >>> 177 } >>> It looks like writing at the addr 0x100000040ca8 in line 172 caused the >>> fault. Why? >>> >>> And then the 2nd page fault in the gdb backtrace as the 1st one was >>> being handled (not sure if that is a bug or just a state of loading of a >>> program). >>> >>> 981 if (dynamic_exists(DT_HASH)) { >>> >>> 982 auto hashtab = dynamic_ptr<Elf64_Word>(DT_HASH); >>> >>> 983 return hashtab[1]; >>> >>> 984 } >>> Is something wrong with the elf files cpiod.so, mkfs.so or zfs.so or >>> something? >>> >>> Can you try to do the same with ROFS? >>> >>> fs=rofs >>> On Saturday, December 5, 2020 at 5:44:12 PM UTC-5 Matthew Kenigsberg >>> wrote: >>> >>>> Struggling to get scripts/build to run on NixOS because I'm getting a >>>> page fault. NixOS does keep shared libraries in nonstandard locations, not >>>> sure if that's breaking something. More details below, but any ideas? >>>> >>>> As far as I can tell, the error is caused by tools/mkfs/mkfs.cc:71: >>>> run_cmd("/zpool.so", zpool_args); >>>> >>>> The error from scripts/build: >>>> >>>> OSv v0.55.0-145-g97f17a7a >>>> eth0: 192.168.122.15 >>>> Booted up in 154.38 ms >>>> Cmdline: /tools/mkfs.so; /tools/cpiod.so --prefix /zfs/zfs/; /zfs.so >>>> set compression=off osv >>>> Running mkfs... >>>> page fault outside application, addr: 0x0000100000040ca8 >>>> [registers] >>>> RIP: 0x000000004039c25a >>>> <elf::object::arch_relocate_jump_slot(elf::symbol_module&, void*, long)+26> >>>> RFL: 0x0000000000010202 CS: 0x0000000000000008 SS: >>>> 0x0000000000000010 >>>> RAX: 0x000010000007a340 RBX: 0x0000100000040ca8 RCX: >>>> 0x000010000006abb0 RDX: 0x0000000000000002 >>>> RSI: 0x00002000001f6f70 RDI: 0xffffa00001058c00 RBP: >>>> 0x00002000001f6f30 R8: 0xffffa00000a68460 >>>> R9: 0xffffa00000f18da0 R10: 0x0000000000000000 R11: >>>> 0x00000000409dd380 R12: 0xffffa00000f18c00 >>>> R13: 0xffffa00000f18da0 R14: 0x0000000000000000 R15: >>>> 0x00000000409dd380 RSP: 0x00002000001f6f20 >>>> Aborted >>>> >>>> [backtrace] >>>> 0x00000000403458d3 <???+1077172435> >>>> 0x00000000403477ce <mmu::vm_fault(unsigned long, exception_frame*)+350> >>>> 0x0000000040398ba2 <page_fault+162> >>>> 0x0000000040397a16 <???+1077508630> >>>> 0x0000000040360a13 <elf::object::resolve_pltgot(unsigned int)+387> >>>> 0x0000000040360c38 <elf_resolve_pltgot+56> >>>> 0x000000004039764f <???+1077507663> >>>> 0xffffa000012b880f <???+19630095> >>>> >>>> Trying to get a backtrace after connecting with gdb: >>>> (gdb) bt >>>> #0 abort (fmt=fmt@entry=0x40644b90 "Assertion failed: %s (%s: %s: >>>> %d)\n") at runtime.cc:105 >>>> #1 0x000000004023c6fb in __assert_fail (expr=expr@entry=0x40672cf8 >>>> "ef->rflags & processor::rflags_if", >>>> file=file@entry=0x40672d25 "arch/x64/mmu.cc", line=line@entry=38, >>>> func=func@entry=0x40672d1a "page_fault") >>>> at runtime.cc:139 >>>> #2 0x0000000040398c05 in page_fault (ef=0xffff800000015048) at >>>> arch/x64/arch-cpu.hh:107 >>>> #3 <signal handler called> >>>> #4 0x000000004035c879 in elf::object::symtab_len >>>> (this=0xffffa00000f18c00) at core/elf.cc:983 >>>> #5 0x000000004035c938 in elf::object::lookup_addr >>>> (this=0xffffa00000f18c00, addr=addr@entry=0x1000000254ce) >>>> at core/elf.cc:1015 >>>> #6 0x000000004035cb07 in elf::program::<lambda(const >>>> elf::program::modules_list&)>::operator() ( >>>> __closure=<synthetic pointer>, __closure=<synthetic pointer>, >>>> ml=...) at core/elf.cc:1620 >>>> #7 elf::program::with_modules<elf::program::lookup_addr(void >>>> const*)::<lambda(const elf::program::modules_list&)> > >>>> (f=..., this=0xffffa00000097e70) at include/osv/elf.hh:702 >>>> #8 elf::program::lookup_addr (this=0xffffa00000097e70, >>>> addr=addr@entry=0x1000000254ce) >>>> at core/elf.cc:1617 >>>> #9 0x00000000404357cc in osv::lookup_name_demangled >>>> (addr=addr@entry=0x1000000254ce, >>>> >>>> buf=buf@entry=0xffff8000012146d0 "???+19630095", len=len@entry=1024) >>>> at core/demangle.cc:47 >>>> #10 0x000000004023c4e0 in print_backtrace () at runtime.cc:85 >>>> #11 0x000000004023c6b4 in abort (fmt=fmt@entry=0x40644a9f "Aborted\n") >>>> at runtime.cc:121 >>>> #12 0x0000000040202989 in abort () at runtime.cc:98 >>>> #13 0x00000000403458d4 in mmu::vm_sigsegv (ef=0xffff800001215068, >>>> addr=<optimized out>) at core/mmu.cc:1314 >>>> #14 mmu::vm_sigsegv (addr=<optimized out>, ef=0xffff800001215068) at >>>> core/mmu.cc:1308 >>>> #15 0x00000000403477cf in mmu::vm_fault (addr=addr@entry=17592186309800, >>>> ef=ef@entry=0xffff800001215068) >>>> at core/mmu.cc:1328 >>>> #16 0x0000000040398ba3 in page_fault (ef=0xffff800001215068) at >>>> arch/x64/mmu.cc:42 >>>> #17 <signal handler called> >>>> #18 0x000000004039c25a in elf::object::arch_relocate_jump_slot >>>> (this=this@entry=0xffffa00000f18c00, sym=..., >>>> addr=addr@entry=0x100000040ca8, addend=addend@entry=0) at >>>> arch/x64/arch-elf.cc:172 >>>> #19 0x0000000040360a14 in elf::object::resolve_pltgot >>>> (this=0xffffa00000f18c00, index=<optimized out>) >>>> at core/elf.cc:843 >>>> #20 0x0000000040360c39 in elf_resolve_pltgot (index=308, >>>> obj=0xffffa00000f18c00) at core/elf.cc:1860 >>>> #21 0x0000000040397650 in __elf_resolve_pltgot () at >>>> arch/x64/elf-dl.S:47 >>>> #22 0x00001000000254cf in ?? () >>>> #23 0xffffa000012b8800 in ?? () >>>> #24 0x00002000001f74a0 in ?? () >>>> #25 0x00001000000254cf in ?? () >>>> #26 0x00002000001f7480 in ?? () >>>> #27 0x00000000403f241c in calloc (nmemb=<optimized out>, >>>> size=<optimized out>) at core/mempool.cc:1811 >>>> #28 0xffff900000a98000 in ?? () >>>> #29 0x0000000000000000 in ?? () >>>> On Saturday, November 28, 2020 at 1:39:46 PM UTC-7 Matthew Kenigsberg >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'll send something, might take a bit before I find time to work on it >>>>> though. >>>>> >>>>> Thanks, >>>>> Matthew >>>>> >>>>> On Saturday, November 28, 2020 at 1:11:11 PM UTC-7 Roman Shaposhnik >>>>> wrote: >>>>> >>>>>> On Tue, Nov 24, 2020 at 8:03 AM Waldek Kozaczuk <[email protected]> >>>>>> wrote: >>>>>> > >>>>>> > Hey, >>>>>> > >>>>>> > Send a patch with a new app that could demonstrate it, please, if >>>>>> you can. I would like to see it. Sounds like a nice improvement. >>>>>> >>>>>> FWIW: I'd love to see it too -- been meaning to play with Nix and >>>>>> this >>>>>> gives me a perfect excuse ;-) >>>>>> >>>>>> Thanks, >>>>>> Roman. >>>>>> >>>>> -- > You received this message because you are subscribed to a topic in the > Google Groups "OSv Development" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/osv-dev/rhjHPr7OBEw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/7913b79b-6c06-4f2a-95d3-9dc44e45eb45n%40googlegroups.com > <https://groups.google.com/d/msgid/osv-dev/7913b79b-6c06-4f2a-95d3-9dc44e45eb45n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CAL9cFfNz2igVaTQo%3DZHGNJ36nRAoXgFGG%2B4o3-5YnLb2pBbwsw%40mail.gmail.com.
