I wonder if we have a bug in core/elf.cc::fix_permissions() or logic around. And we might be making the wrong part of the mapping readable based on GNU_RELRO header. I wonder if you are able to create ZFS image by temporarily commenting out the line 1429 of core/elf.cc:
ef->fix_permissions(); Also, would it possible to get copies of those binaries: /libenviron.so: libenviron.so /libvdso.so: libvdso.so /zpool.so: zpool.so /libzfs.so: libzfs.so /libuutil.so: libuutil.so /zfs.so: zfs.so /tools/mkfs.so: tools/mkfs/mkfs.so /tools/cpiod.so: tools/cpiod/cpiod.so /tools/mount-fs.so: tools/mount/mount-fs.so /tools/umount.so: tools/mount/umount.so /usr/lib/libgcc_s.so.1: %(libgcc_s_dir)s/libgcc_s.so.1 Ideally, the stripped versions. That would help me to re-create the problem and investigate further. On Tuesday, December 8, 2020 at 1:25:10 PM UTC-5 Matthew Kenigsberg wrote: > [nix-shell:~/osv]$ readelf -l build/release/libzfs-stripped.so > > Elf file type is DYN (Shared object file) > Entry point 0xc8f0 > There are 8 program headers, starting at offset 64 > > Program Headers: > Type Offset VirtAddr PhysAddr > FileSiz MemSiz Flags Align > LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x000000000000a9b8 0x000000000000a9b8 R 0x1000 > LOAD 0x000000000000b000 0x000000000000b000 0x000000000000b000 > 0x000000000001e0a9 0x000000000001e0a9 R E 0x1000 > LOAD 0x000000000002a000 0x000000000002a000 0x000000000002a000 > 0x00000000000093a0 0x00000000000093a0 R 0x1000 > LOAD 0x0000000000034010 0x0000000000035010 0x0000000000035010 > 0x0000000000001810 0x0000000000002c20 RW 0x1000 > DYNAMIC 0x00000000000340e0 0x00000000000350e0 0x00000000000350e0 > 0x0000000000000210 0x0000000000000210 RW 0x8 > GNU_EH_FRAME 0x000000000002e768 0x000000000002e768 0x000000000002e768 > 0x0000000000000d04 0x0000000000000d04 R 0x4 > GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000 RW 0x10 > GNU_RELRO 0x0000000000034010 0x0000000000035010 0x0000000000035010 > 0x0000000000000ff0 0x0000000000000ff0 R 0x1 > > Section to Segment mapping: > Segment Sections... > 00 .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn > .rela.plt > 01 .init .plt .plt.got .text .fini > 02 .rodata .eh_frame_hdr .eh_frame > 03 .init_array .fini_array .data.rel.ro .dynamic .got .data .bss > 04 .dynamic > 05 .eh_frame_hdr > 06 > 07 .init_array .fini_array .data.rel.ro .dynamic .got > > On Tuesday, December 8, 2020 at 11:17:46 AM UTC-7 [email protected] > wrote: > >> Back to why it is failing. >> >> Based on what you sent us: >> .. >> 0x000010000000b000 0x0000100000016000 [44.0 kB] flags=fmF perm=r >> offset=0x00000000 path=/libzfs.so >> 0x0000100000016000 0x0000100000035000 [124.0 kB] flags=fmF perm=rx >> offset=0x0000b000 path=/libzfs.so >> 0x0000100000035000 0x000010000003f000 [40.0 kB] flags=fmF perm=r >> offset=0x0002a000 path=/libzfs.so >> >> *0x0000100000040000 0x0000100000041000 [4.0 kB] flags=fmF perm=r >> offset=0x00034000 path=/libzfs.so*0x0000100000041000 0x0000100000042000 >> [4.0 kB] flags=fmF perm=rw offset=0x00035000 path=/libzfs.so >> .. >> >> The page fault in arch_relocate_jump_slot() is caused by an attempt to >> write at the address 0x100000040ca8 which falls into a 4th mapping range >> from the above which can only be read from. So that is the permission >> fault. The question is why the address is in that range? The address should >> be somewhere in GOT in the 5th range - 0x0000100000041000 >> 0x0000100000042000 [4.0 kB] flags=fmF perm=rw offset=0x00035000 >> path=/libzfs.so which has read/write permission. >> >> On Ubuntu host when I run the same command like and add extra debugging >> to print statements like this: >> >> ELF [tid:36, mod:5, /*libzfs*.so]: arch_relocate_jump_slot, >> addr:0x10000007a8d0 >> They all print addresses within the range 0x10000007a000 - 0x10000007b000 >> which are read-write permitted as they should be: >> 0x0000100000044000 0x000010000004e000 [40.0 kB] flags=fmF >> perm=r offset=0x00000000 path=/libzfs.so >> 0x000010000004e000 0x000010000006f000 [132.0 kB] flags=fmF >> perm=rx offset=0x0000a000 path=/libzfs.so >> 0x000010000006f000 0x0000100000079000 [40.0 kB] flags=fmF >> perm=r offset=0x0002b000 path=/libzfs.so >> 0x0000100000079000 0x000010000007a000 [4.0 kB] flags=fmF >> perm=r offset=0x00034000 path=/libzfs.so >> *0x000010000007a000 0x000010000007c000 [8.0 kB] flags=fmF >> perm=rw offset=0x00035000 path=/libzfs.so * >> >> I wonder if we have a bug when calculating where each segment should be >> mapped: >> >> 400 void file::load_segment(const Elf64_Phdr& phdr) >> 401 { >> 402 ulong vstart = align_down(phdr.p_vaddr, mmu::page_size); >> 403 ulong filesz_unaligned = phdr.p_vaddr + phdr.p_filesz - vstart; >> 404 ulong filesz = align_up(filesz_unaligned, mmu::page_size); >> 405 ulong memsz = align_up(phdr.p_vaddr + phdr.p_memsz, >> mmu::page_size) - vstart; >> 406 >> 407 unsigned perm = get_segment_mmap_permissions(phdr); >> 408 >> 409 auto flag = mmu::mmap_fixed | (mlocked() ? mmu::mmap_populate : >> 0); >> 410 mmu::map_file(_base + vstart, filesz, flag, perm, _f, >> align_down(phdr.p_offset, mmu::page_size)); >> 411 if (phdr.p_filesz != phdr.p_memsz) { >> 412 assert(perm & mmu::perm_write); >> 413 memset(_base + vstart + filesz_unaligned, 0, filesz - >> filesz_unaligned); >> 414 if (memsz != filesz) { >> 415 mmu::map_anon(_base + vstart + filesz, memsz - filesz, >> flag, perm); >> 416 } >> 417 } >> 418 elf_debug("Loaded and mapped PT_LOAD segment at: %018p of size: >> 0x%x\n", _base + vstart, filesz); >> 419 } >> >> BTW I am also interested what the output of this readelf command for your >> libzfs.so is. Mine is this: >> >> readelf -l build/release/libzfs-stripped.so >> >> Elf file type is DYN (Shared object file) >> Entry point 0xd1d0 >> There are 11 program headers, starting at offset 64 >> >> Program Headers: >> Type Offset VirtAddr PhysAddr >> FileSiz MemSiz Flags Align >> LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000 >> 0x00000000000098e0 0x00000000000098e0 R 0x1000 >> LOAD 0x000000000000a000 0x000000000000a000 0x000000000000a000 >> 0x00000000000201a1 0x00000000000201a1 R E 0x1000 >> LOAD 0x000000000002b000 0x000000000002b000 0x000000000002b000 >> 0x0000000000009258 0x0000000000009258 R 0x1000 >> LOAD 0x0000000000034cb0 0x0000000000035cb0 0x0000000000035cb0 >> 0x00000000000017f0 0x0000000000002c00 RW 0x1000 >> DYNAMIC 0x0000000000034d80 0x0000000000035d80 0x0000000000035d80 >> 0x00000000000001d0 0x00000000000001d0 RW 0x8 >> NOTE 0x00000000000002a8 0x00000000000002a8 0x00000000000002a8 >> 0x0000000000000020 0x0000000000000020 R 0x8 >> NOTE 0x00000000000002c8 0x00000000000002c8 0x00000000000002c8 >> 0x0000000000000024 0x0000000000000024 R 0x4 >> GNU_PROPERTY 0x00000000000002a8 0x00000000000002a8 0x00000000000002a8 >> 0x0000000000000020 0x0000000000000020 R 0x8 >> GNU_EH_FRAME 0x000000000002f768 0x000000000002f768 0x000000000002f768 >> 0x0000000000000cec 0x0000000000000cec R 0x4 >> GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 >> 0x0000000000000000 0x0000000000000000 RW 0x10 >> GNU_RELRO 0x0000000000034cb0 0x0000000000035cb0 0x0000000000035cb0 >> 0x0000000000000350 0x0000000000000350 R 0x1 >> >> Section to Segment mapping: >> Segment Sections... >> 00 .note.gnu.property .note.gnu.build-id .gnu.hash .dynsym .dynstr >> .gnu.version .gnu.version_r .rela.dyn .rela.plt >> 01 .init .plt .plt.got .plt.sec .text .fini >> 02 .rodata .eh_frame_hdr .eh_frame >> 03 .init_array .fini_array .data.rel.ro .dynamic .got .got.plt >> .data .bss >> 04 .dynamic >> 05 .note.gnu.property >> 06 .note.gnu.build-id >> 07 .note.gnu.property >> 08 .eh_frame_hdr >> 09 >> 10 .init_array .fini_array .data.rel.ro .dynamic .got >> >> On Tuesday, December 8, 2020 at 11:39:43 AM UTC-5 Matthew Kenigsberg >> wrote: >> >>> My gdb is not the strongest but if I hbreak on arch_relocate_jump_slot >>> looks like _pathname is /libzfs.so, eventually /zpool.so, and then a single >>> /libzfs.so before continue hangs >>> >>> On Tuesday, December 8, 2020 at 9:11:15 AM UTC-7 Matthew Kenigsberg >>> wrote: >>> >>>> Nix is a package manager, and NixOS is an operating system built >>>> completely around the package manager. So every library is stored >>>> somewhere >>>> in /nix/store, like for example on Nix there is never anything like >>>> /lib64/ld-linux-x86-64.so. >>>> It would be /nix/store/.../ld-linux-x86-64.so. I could install the package >>>> manager on a different OS, in which case I might have both /lib64 and >>>> /nix/store, but on NixOS I'll just have the latter. Does that make sense? >>>> Not >>>> sure if that's messing up something with linking. Guessing I can't >>>> reproduce the error on any other OS, but happy to try. >>>> On Tuesday, December 8, 2020 at 9:05:11 AM UTC-7 Matthew Kenigsberg >>>> wrote: >>>> >>>>> (gdb) connect >>>>> abort (fmt=fmt@entry=0x40645bf0 "Assertion failed: %s (%s: %s: %d)\n") >>>>> at runtime.cc:105 >>>>> 105 do {} while (true); >>>>> (gdb) osv syms >>>>> manifest.find_file: path=/libvdso.so, found file=libvdso.so >>>>> /home/matthew/osv/build/release.x64/libvdso.so 0x100000000000 >>>>> add symbol table from file >>>>> "/home/matthew/osv/build/release.x64/libvdso.so" at >>>>> .text_addr = 0x100000001040 >>>>> .hash_addr = 0x1000000001c8 >>>>> .gnu.hash_addr = 0x100000000200 >>>>> .dynsym_addr = 0x100000000238 >>>>> .dynstr_addr = 0x1000000002f8 >>>>> .gnu.version_addr = 0x1000000003be >>>>> .gnu.version_d_addr = 0x1000000003d0 >>>>> .rela.plt_addr = 0x100000000408 >>>>> .plt_addr = 0x100000001000 >>>>> .eh_frame_addr = 0x100000002000 >>>>> .dynamic_addr = 0x100000003e60 >>>>> .got_addr = 0x100000003fd0 >>>>> .comment_addr = 0x100000000000 >>>>> .debug_aranges_addr = 0x100000000000 >>>>> .debug_info_addr = 0x100000000000 >>>>> .debug_abbrev_addr = 0x100000000000 >>>>> .debug_line_addr = 0x100000000000 >>>>> .debug_str_addr = 0x100000000000 >>>>> .debug_loc_addr = 0x100000000000 >>>>> .symtab_addr = 0x100000000000 >>>>> .strtab_addr = 0x100000000000 >>>>> warning: section .comment not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .debug_aranges not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .debug_info not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .debug_abbrev not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .debug_line not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .debug_str not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .debug_loc not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .symtab not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> warning: section .strtab not found in >>>>> /home/matthew/osv/build/release.x64/libvdso.so >>>>> manifest.find_file: path=/tools/mkfs.so, found file=tools/mkfs/mkfs.so >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so 0x100000004000 >>>>> add symbol table from file >>>>> "/home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so" at >>>>> .text_addr = 0x100000006250 >>>>> .hash_addr = 0x100000004200 >>>>> .gnu.hash_addr = 0x100000004360 >>>>> .dynsym_addr = 0x1000000043c0 >>>>> .dynstr_addr = 0x100000004840 >>>>> .gnu.version_addr = 0x100000005092 >>>>> .gnu.version_r_addr = 0x1000000050f8 >>>>> .rela.dyn_addr = 0x100000005148 >>>>> .rela.plt_addr = 0x100000005298 >>>>> .init_addr = 0x100000006000 >>>>> .plt_addr = 0x100000006020 >>>>> .plt.got_addr = 0x100000006240 >>>>> .fini_addr = 0x10000000737c >>>>> .rodata_addr = 0x100000008000 >>>>> .eh_frame_hdr_addr = 0x10000000817c >>>>> .eh_frame_addr = 0x100000008210 >>>>> .gcc_except_table_addr = 0x100000008530 >>>>> .init_array_addr = 0x100000009c60 >>>>> .fini_array_addr = 0x100000009c70 >>>>> .dynamic_addr = 0x100000009c78 >>>>> .got_addr = 0x100000009e98 >>>>> .data_addr = 0x10000000a000 >>>>> .bss_addr = 0x10000000a010 >>>>> .comment_addr = 0x100000004000 >>>>> .debug_aranges_addr = 0x100000004000 >>>>> .debug_info_addr = 0x100000004000 >>>>> .debug_abbrev_addr = 0x100000004000 >>>>> --Type <RET> for more, q to quit, c to continue without paging--c >>>>> .debug_line_addr = 0x100000004000 >>>>> .debug_str_addr = 0x100000004000 >>>>> .debug_loc_addr = 0x100000004000 >>>>> .debug_ranges_addr = 0x100000004000 >>>>> .symtab_addr = 0x100000004000 >>>>> .strtab_addr = 0x100000004000 >>>>> warning: section .comment not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .debug_aranges not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .debug_info not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .debug_abbrev not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .debug_line not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .debug_str not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .debug_loc not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .debug_ranges not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .symtab not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> warning: section .strtab not found in >>>>> /home/matthew/osv/build/release.x64/tools/mkfs/mkfs.so >>>>> manifest.find_file: path=/libzfs.so, found file=libzfs.so >>>>> /home/matthew/osv/build/release.x64/libzfs.so 0x10000000b000 >>>>> add symbol table from file >>>>> "/home/matthew/osv/build/release.x64/libzfs.so" at >>>>> .text_addr = 0x1000000178f0 >>>>> .hash_addr = 0x10000000b200 >>>>> .gnu.hash_addr = 0x10000000c318 >>>>> .dynsym_addr = 0x10000000cd78 >>>>> .dynstr_addr = 0x100000010300 >>>>> .gnu.version_addr = 0x1000000124e2 >>>>> .gnu.version_r_addr = 0x100000012958 >>>>> .rela.dyn_addr = 0x1000000129b8 >>>>> .rela.plt_addr = 0x1000000134b0 >>>>> .init_addr = 0x100000016000 >>>>> .plt_addr = 0x100000016020 >>>>> .plt.got_addr = 0x1000000178e0 >>>>> .fini_addr = 0x1000000340a0 >>>>> .rodata_addr = 0x100000035000 >>>>> .eh_frame_hdr_addr = 0x100000039768 >>>>> .eh_frame_addr = 0x10000003a470 >>>>> .init_array_addr = 0x100000040010 >>>>> .fini_array_addr = 0x100000040018 >>>>> .data.rel.ro_addr = 0x100000040020 >>>>> .dynamic_addr = 0x1000000400e0 >>>>> .got_addr = 0x1000000402f0 >>>>> .data_addr = 0x100000041000 >>>>> .bss_addr = 0x100000041820 >>>>> .comment_addr = 0x10000000b000 >>>>> .debug_aranges_addr = 0x10000000b000 >>>>> .debug_info_addr = 0x10000000b000 >>>>> .debug_abbrev_addr = 0x10000000b000 >>>>> .debug_line_addr = 0x10000000b000 >>>>> .debug_str_addr = 0x10000000b000 >>>>> .debug_loc_addr = 0x10000000b000 >>>>> .debug_ranges_addr = 0x10000000b000 >>>>> .symtab_addr = 0x10000000b000 >>>>> .strtab_addr = 0x10000000b000 >>>>> warning: section .comment not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .debug_aranges not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .debug_info not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .debug_abbrev not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .debug_line not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .debug_str not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .debug_loc not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .debug_ranges not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .symtab not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> warning: section .strtab not found in >>>>> /home/matthew/osv/build/release.x64/libzfs.so >>>>> manifest.find_file: path=/libuutil.so, found file=libuutil.so >>>>> /home/matthew/osv/build/release.x64/libuutil.so 0x100000043000 >>>>> add symbol table from file >>>>> "/home/matthew/osv/build/release.x64/libuutil.so" at >>>>> .text_addr = 0x1000000463c0 >>>>> .hash_addr = 0x100000043200 >>>>> .gnu.hash_addr = 0x100000043640 >>>>> .dynsym_addr = 0x1000000438f8 >>>>> .dynstr_addr = 0x100000044600 >>>>> .gnu.version_addr = 0x100000044da4 >>>>> .gnu.version_r_addr = 0x100000044ec0 >>>>> .rela.dyn_addr = 0x100000044f00 >>>>> .rela.plt_addr = 0x100000045068 >>>>> .init_addr = 0x100000046000 >>>>> .plt_addr = 0x100000046020 >>>>> .plt.got_addr = 0x1000000463b0 >>>>> .fini_addr = 0x100000049adc >>>>> .rodata_addr = 0x10000004a000 >>>>> .eh_frame_hdr_addr = 0x10000004ac84 >>>>> .eh_frame_addr = 0x10000004af90 >>>>> .init_array_addr = 0x10000004dbd8 >>>>> .fini_array_addr = 0x10000004dbe0 >>>>> .dynamic_addr = 0x10000004dbe8 >>>>> .got_addr = 0x10000004dde8 >>>>> .data_addr = 0x10000004e000 >>>>> .bss_addr = 0x10000004e260 >>>>> .comment_addr = 0x100000043000 >>>>> .debug_aranges_addr = 0x100000043000 >>>>> .debug_info_addr = 0x100000043000 >>>>> .debug_abbrev_addr = 0x100000043000 >>>>> .debug_line_addr = 0x100000043000 >>>>> .debug_str_addr = 0x100000043000 >>>>> .debug_loc_addr = 0x100000043000 >>>>> .debug_ranges_addr = 0x100000043000 >>>>> .symtab_addr = 0x100000043000 >>>>> .strtab_addr = 0x100000043000 >>>>> warning: section .comment not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .debug_aranges not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .debug_info not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .debug_abbrev not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .debug_line not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .debug_str not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .debug_loc not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .debug_ranges not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .symtab not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> warning: section .strtab not found in >>>>> /home/matthew/osv/build/release.x64/libuutil.so >>>>> manifest.find_file: path=/usr/lib/libgcc_s.so.1, found >>>>> file=%(libgcc_s_dir)s/libgcc_s.so.1 >>>>> ERROR: Unable to locate object file for: /usr/lib/libgcc_s.so.1 >>>>> 0x10000004f000 >>>>> manifest.find_file: path=/zpool.so, found file=zpool.so >>>>> /home/matthew/osv/build/release.x64/zpool.so 0x100000069000 >>>>> add symbol table from file >>>>> "/home/matthew/osv/build/release.x64/zpool.so" at >>>>> .text_addr = 0x10000006ebc0 >>>>> .hash_addr = 0x100000069200 >>>>> .gnu.hash_addr = 0x100000069868 >>>>> .dynsym_addr = 0x100000069968 >>>>> .dynstr_addr = 0x10000006ad18 >>>>> .gnu.version_addr = 0x10000006b958 >>>>> .gnu.version_r_addr = 0x10000006bb00 >>>>> .rela.dyn_addr = 0x10000006bb30 >>>>> .rela.plt_addr = 0x10000006c208 >>>>> .init_addr = 0x10000006e000 >>>>> .plt_addr = 0x10000006e020 >>>>> .plt.got_addr = 0x10000006ebb0 >>>>> .fini_addr = 0x10000007a420 >>>>> .rodata_addr = 0x10000007b000 >>>>> .eh_frame_hdr_addr = 0x10000007fd30 >>>>> .eh_frame_addr = 0x100000080028 >>>>> .init_array_addr = 0x100000082758 >>>>> .fini_array_addr = 0x100000082760 >>>>> .dynamic_addr = 0x100000082768 >>>>> .got_addr = 0x100000082978 >>>>> .data_addr = 0x100000083000 >>>>> .bss_addr = 0x100000083360 >>>>> .comment_addr = 0x100000069000 >>>>> .debug_aranges_addr = 0x100000069000 >>>>> .debug_info_addr = 0x100000069000 >>>>> .debug_abbrev_addr = 0x100000069000 >>>>> .debug_line_addr = 0x100000069000 >>>>> .debug_str_addr = 0x100000069000 >>>>> .debug_loc_addr = 0x100000069000 >>>>> .debug_ranges_addr = 0x100000069000 >>>>> .symtab_addr = 0x100000069000 >>>>> .strtab_addr = 0x100000069000 >>>>> warning: section .comment not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .debug_aranges not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .debug_info not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .debug_abbrev not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .debug_line not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .debug_str not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .debug_loc not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .debug_ranges not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .symtab not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> warning: section .strtab not found in >>>>> /home/matthew/osv/build/release.x64/zpool.so >>>>> (gdb) osv mmap >>>>> 0x0000000000000000 0x0000000000000000 [0.0 kB] flags=none perm=none >>>>> 0x0000100000000000 0x0000100000001000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00000000 path=/libvdso.so >>>>> 0x0000100000001000 0x0000100000002000 [4.0 kB] flags=fmF perm=rx >>>>> offset=0x00001000 path=/libvdso.so >>>>> 0x0000100000002000 0x0000100000003000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00002000 path=/libvdso.so >>>>> 0x0000100000003000 0x0000100000004000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00002000 path=/libvdso.so >>>>> 0x0000100000004000 0x0000100000006000 [8.0 kB] flags=fmF perm=r >>>>> offset=0x00000000 path=/tools/mkfs.so >>>>> 0x0000100000006000 0x0000100000008000 [8.0 kB] flags=fmF perm=rx >>>>> offset=0x00002000 path=/tools/mkfs.so >>>>> 0x0000100000008000 0x0000100000009000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00004000 path=/tools/mkfs.so >>>>> 0x0000100000009000 0x000010000000a000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00004000 path=/tools/mkfs.so >>>>> 0x000010000000a000 0x000010000000b000 [4.0 kB] flags=fmF perm=rw >>>>> offset=0x00005000 path=/tools/mkfs.so >>>>> 0x000010000000b000 0x0000100000016000 [44.0 kB] flags=fmF perm=r >>>>> offset=0x00000000 path=/libzfs.so >>>>> 0x0000100000016000 0x0000100000035000 [124.0 kB] flags=fmF perm=rx >>>>> offset=0x0000b000 path=/libzfs.so >>>>> 0x0000100000035000 0x000010000003f000 [40.0 kB] flags=fmF perm=r >>>>> offset=0x0002a000 path=/libzfs.so >>>>> 0x0000100000040000 0x0000100000041000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00034000 path=/libzfs.so >>>>> 0x0000100000041000 0x0000100000042000 [4.0 kB] flags=fmF perm=rw >>>>> offset=0x00035000 path=/libzfs.so >>>>> 0x0000100000042000 0x0000100000043000 [4.0 kB] flags=f perm=rw >>>>> 0x0000100000043000 0x0000100000046000 [12.0 kB] flags=fmF perm=r >>>>> offset=0x00000000 path=/libuutil.so >>>>> 0x0000100000046000 0x000010000004a000 [16.0 kB] flags=fmF perm=rx >>>>> offset=0x00003000 path=/libuutil.so >>>>> 0x000010000004a000 0x000010000004c000 [8.0 kB] flags=fmF perm=r >>>>> offset=0x00007000 path=/libuutil.so >>>>> 0x000010000004d000 0x000010000004e000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00009000 path=/libuutil.so >>>>> 0x000010000004e000 0x000010000004f000 [4.0 kB] flags=fmF perm=rw >>>>> offset=0x0000a000 path=/libuutil.so >>>>> 0x000010000004f000 0x0000100000052000 [12.0 kB] flags=fmF perm=r >>>>> offset=0x00000000 path=/usr/lib/libgcc_s.so.1 >>>>> 0x0000100000052000 0x0000100000063000 [68.0 kB] flags=fmF perm=rx >>>>> offset=0x00003000 path=/usr/lib/libgcc_s.so.1 >>>>> 0x0000100000063000 0x0000100000067000 [16.0 kB] flags=fmF perm=r >>>>> offset=0x00014000 path=/usr/lib/libgcc_s.so.1 >>>>> 0x0000100000067000 0x0000100000068000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00017000 path=/usr/lib/libgcc_s.so.1 >>>>> 0x0000100000068000 0x0000100000069000 [4.0 kB] flags=fmF perm=rw >>>>> offset=0x00018000 path=/usr/lib/libgcc_s.so.1 >>>>> 0x0000100000069000 0x000010000006e000 [20.0 kB] flags=fmF perm=r >>>>> offset=0x00000000 path=/zpool.so >>>>> 0x000010000006e000 0x000010000007b000 [52.0 kB] flags=fmF perm=rx >>>>> offset=0x00005000 path=/zpool.so >>>>> 0x000010000007b000 0x0000100000082000 [28.0 kB] flags=fmF perm=r >>>>> offset=0x00012000 path=/zpool.so >>>>> 0x0000100000082000 0x0000100000083000 [4.0 kB] flags=fmF perm=r >>>>> offset=0x00018000 path=/zpool.so >>>>> 0x0000100000083000 0x0000100000084000 [4.0 kB] flags=fmF perm=rw >>>>> offset=0x00019000 path=/zpool.so >>>>> 0x0000100000084000 0x0000100000086000 [8.0 kB] flags=f perm=rw >>>>> 0x0000200000000000 0x0000200000001000 [4.0 kB] flags=p perm=none >>>>> 0x0000200000001000 0x0000200000002000 [4.0 kB] flags=p perm=none >>>>> 0x0000200000002000 0x0000200000101000 [1020.0 kB] flags=p perm=rw >>>>> 0x0000200000101000 0x0000200000102000 [4.0 kB] flags=p perm=none >>>>> 0x0000200000102000 0x0000200000201000 [1020.0 kB] flags=p perm=rw >>>>> 0x0000800000000000 0x0000800000000000 [0.0 kB] flags=none perm=none >>>>> (gdb) bt >>>>> #0 abort (fmt=fmt@entry=0x40645bf0 "Assertion failed: %s (%s: %s: >>>>> %d)\n") at runtime.cc:105 >>>>> #1 0x000000004023c75b in __assert_fail (expr=expr@entry=0x406742f8 >>>>> "ef->rflags & processor::rflags_if", file=file@entry=0x40674325 >>>>> "arch/x64/mmu.cc", line=line@entry=38, func=func@entry=0x4067431a >>>>> "page_fault") at runtime.cc:139 >>>>> #2 0x0000000040399305 in page_fault (ef=0xffff800000015048) at >>>>> arch/x64/arch-cpu.hh:107 >>>>> #3 <signal handler called> >>>>> #4 0x000000004035ca19 in elf::object::symtab_len >>>>> (this=0xffffa00000f19a00) at core/elf.cc:983 >>>>> #5 0x000000004035cad8 in elf::object::lookup_addr >>>>> (this=0xffffa00000f19a00, addr=addr@entry=0x1000000254ce <is_mounted+30>) >>>>> at core/elf.cc:1015 >>>>> #6 0x000000004035cca7 in elf::program::<lambda(const >>>>> elf::program::modules_list&)>::operator() (__closure=<synthetic pointer>, >>>>> __closure=<synthetic pointer>, ml=...) at core/elf.cc:1620 >>>>> #7 elf::program::with_modules<elf::program::lookup_addr(void >>>>> const*)::<lambda(const elf::program::modules_list&)> > (f=..., >>>>> this=0xffffa00000097e70) at include/osv/elf.hh:702 >>>>> #8 elf::program::lookup_addr (this=0xffffa00000097e70, >>>>> addr=addr@entry=0x1000000254ce <is_mounted+30>) at core/elf.cc:1617 >>>>> #9 0x00000000404367cc in osv::lookup_name_demangled >>>>> (addr=addr@entry=0x1000000254ce <is_mounted+30>, >>>>> buf=buf@entry=0xffff8000012156d0 "???+19929103", len=len@entry=1024) at >>>>> core/demangle.cc:47 >>>>> #10 0x000000004023c540 in print_backtrace () at runtime.cc:85 >>>>> #11 0x000000004023c714 in abort (fmt=fmt@entry=0x40645aff "Aborted\n") >>>>> at runtime.cc:121 >>>>> #12 0x0000000040202989 in abort () at runtime.cc:98 >>>>> #13 0x0000000040345934 in mmu::vm_sigsegv (ef=0xffff800001216068, >>>>> addr=<optimized out>) at core/mmu.cc:1314 >>>>> #14 mmu::vm_sigsegv (addr=<optimized out>, ef=0xffff800001216068) at >>>>> core/mmu.cc:1308 >>>>> #15 0x000000004034782f in mmu::vm_fault >>>>> (addr=addr@entry=17592186309800, ef=ef@entry=0xffff800001216068) at >>>>> core/mmu.cc:1328 >>>>> #16 0x00000000403992a3 in page_fault (ef=0xffff800001216068) at >>>>> arch/x64/mmu.cc:42 >>>>> #17 <signal handler called> >>>>> #18 0x000000004039c95a in elf::object::arch_relocate_jump_slot >>>>> (this=this@entry=0xffffa00000f19a00, sym=..., >>>>> addr=addr@entry=0x100000040ca8 <[email protected]>, >>>>> addend=addend@entry=0) >>>>> at arch/x64/arch-elf.cc:172 >>>>> #19 0x0000000040361004 in elf::object::resolve_pltgot >>>>> (this=0xffffa00000f19a00, index=<optimized out>) at core/elf.cc:843 >>>>> #20 0x0000000040361229 in elf_resolve_pltgot (index=308, >>>>> obj=0xffffa00000f19a00) at core/elf.cc:1860 >>>>> #21 0x0000000040397d50 in __elf_resolve_pltgot () at >>>>> arch/x64/elf-dl.S:47 >>>>> #22 0x00001000000254cf in is_mounted (zfs_hdl=0x134, >>>>> special=<optimized out>, where=0x403f3377 <malloc(size_t)+71>) at >>>>> bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_mount.c:214 >>>>> #23 0xffff900000a99000 in ?? () >>>>> #24 0x0000000000000000 in ?? () >>>>> >>>>> On Tuesday, December 8, 2020 at 8:53:52 AM UTC-7 [email protected] >>>>> wrote: >>>>> >>>>>> It would be also nice to understand if we are crashing on the 1st >>>>>> arch_relocate_jump_slot() for libfzs.so or is it a specific JUMP_SLOT >>>>>> that >>>>>> causes this crash? >>>>>> >>>>>> On Tuesday, December 8, 2020 at 10:39:06 AM UTC-5 Waldek Kozaczuk >>>>>> wrote: >>>>>> >>>>>>> After you connect with gdb can you run 'osv mmap' and send us the >>>>>>> output. Make sure you run 'osv syms' before it and dump backtrace >>>>>>> after. >>>>>>> Please see >>>>>>> https://github.com/cloudius-systems/osv/wiki/Debugging-OSv for any >>>>>>> details. >>>>>>> >>>>>>> BTW can you build and run OSv ZFS image on the host without NIX? As >>>>>>> I understand NIX is really just a layer on top of any Linux >>>>>>> distribution, >>>>>>> no? I am afraid I do not still understand what exactly NiX is I guess. >>>>>>> >>>>>>> >>>>>>> On Monday, December 7, 2020 at 2:58:40 PM UTC-5 Matthew Kenigsberg >>>>>>> wrote: >>>>>>> >>>>>>>> (gdb) frame 18 >>>>>>>> #18 0x000000004039c95a in elf::object::arch_relocate_jump_slot >>>>>>>> (this=this@entry=0xffffa0000110fa00, sym=..., >>>>>>>> addr=addr@entry=0x100000040ca8, addend=addend@entry=0) at >>>>>>>> arch/x64/arch-elf.cc:172 >>>>>>>> 172 *static_cast<void**>(addr) = sym.relocated_addr(); >>>>>>>> (gdb) print _pathname >>>>>>>> $14 = {static npos = 18446744073709551615, >>>>>>>> _M_dataplus = {<std::allocator<char>> = >>>>>>>> {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data >>>>>>>> fields>}, >>>>>>>> _M_p = 0xffffa0000110fa30 "/libzfs.so"}, _M_string_length = 10, >>>>>>>> { >>>>>>>> _M_local_buf = "/libzfs.so\000\000\000\000\000", >>>>>>>> _M_allocated_capacity = 3347131623889529903}} >>>>>>>> >>>>>>>> Also been wondering if nix using nonstandard paths is causing >>>>>>>> problems, like for libc: >>>>>>>> [nix-shell:~/osv/build/release]$ ldd libzfs.so >>>>>>>> linux-vdso.so.1 (0x00007ffcedbb9000) >>>>>>>> libuutil.so => not found >>>>>>>> libc.so.6 => >>>>>>>> /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6 >>>>>>>> (0x00007f7594f38000) >>>>>>>> >>>>>>>> >>>>>>>> /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib64/ld-linux-x86-64.so.2 >>>>>>>> >>>>>>>> (0x00007f7595131000) >>>>>>>> On Sunday, December 6, 2020 at 8:43:10 AM UTC-7 [email protected] >>>>>>>> wrote: >>>>>>>> >>>>>>>>> It might be easier to simply print '_pathname' value if you switch >>>>>>>>> to the right frame in gdb. It would be nice to confirm that the >>>>>>>>> problem we >>>>>>>>> have is with zpool.so and that might lead to understanding why this >>>>>>>>> crash >>>>>>>>> happens. Maybe the is something wrong with building zpool.so. >>>>>>>>> >>>>>>>>> BTW based on this fragment of the stacktrace: >>>>>>>>> >>>>>>>>> #6 0x000000004035cb07 in elf::program::<lambda(const >>>>>>>>> elf::program::modules_list&)>::operator() ( >>>>>>>>> __closure=<synthetic pointer>, __closure=<synthetic pointer>, >>>>>>>>> ml=...) at core/elf.cc:1620 >>>>>>>>> #7 elf::program::with_modules<elf::program::lookup_addr(void >>>>>>>>> const*)::<lambda(const elf::program::modules_list&)> > >>>>>>>>> (f=..., this=0xffffa00000097e70) at include/osv/elf.hh:702 >>>>>>>>> #8 elf::program::lookup_addr (this=0xffffa00000097e70, >>>>>>>>> addr=addr@entry=0x1000000254ce) at core/elf.cc:1617 >>>>>>>>> #9 0x00000000404357cc in osv::lookup_name_demangled >>>>>>>>> (addr=addr@entry=0x1000000254ce, >>>>>>>>> buf=buf@entry=0xffff8000012146d0 "???+19630095", >>>>>>>>> len=len@entry=1024) at core/demangle.cc:47 >>>>>>>>> #10 0x000000004023c4e0 in print_backtrace () at runtime.cc:85 >>>>>>>>> >>>>>>>>> It seems we have a bug (or need of improvement) in >>>>>>>>> print_backtrace() to make it NOT try to demangle names like >>>>>>>>> "???+19630095" >>>>>>>>> which causes follow-up fault. >>>>>>>>> >>>>>>>>> At the same time, it is strange that we crash at line 983 which >>>>>>>>> seems to indicate something goes wrong when processing zpool.so. >>>>>>>>> >>>>>>>>> 981 if (dynamic_exists(DT_HASH)) { >>>>>>>>> >>>>>>>>> 982 auto hashtab = dynamic_ptr<Elf64_Word>(DT_HASH); >>>>>>>>> >>>>>>>>> *983 return hashtab[1];* >>>>>>>>> >>>>>>>>> 984 } >>>>>>>>> >>>>>>>>> On Sunday, December 6, 2020 at 10:06:21 AM UTC-5 Waldek Kozaczuk >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Can you run the ROFS image you built? Also as I understand it NIX >>>>>>>>>> is a package manager but what Linux distribution are you using? >>>>>>>>>> >>>>>>>>>> As far as ZFS goes could you enable ELF debugging - change this >>>>>>>>>> line: >>>>>>>>>> >>>>>>>>>> conf-debug_elf=0 >>>>>>>>>> >>>>>>>>>> To >>>>>>>>>> >>>>>>>>>> conf-debug_elf=1 >>>>>>>>>> >>>>>>>>>> In conf/base.mk, delete core/elf.o and force rebuild the kernel. >>>>>>>>>> I think you may also need to change the script upload_manifest.py to >>>>>>>>>> peeped >>>>>>>>>> ‘—verbose’ to the command line with cpiod.so >>>>>>>>>> >>>>>>>>>> It should show more info about elf loading. It may still be >>>>>>>>>> necessary to add extra printouts to capture which exact elf it is >>>>>>>>>> crashing >>>>>>>>>> on in arch_relocate_jump(). >>>>>>>>>> >>>>>>>>>> In worst case I would need a copy of your loader-stripped.elf and >>>>>>>>>> possibly all the other files like cpiod.so, zfs.so that go into the >>>>>>>>>> bootfs >>>>>>>>>> part of the image. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Waldek >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sat, Dec 5, 2020 at 19:31 Matthew Kenigsberg < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> After forcing it to use the right path for libz.so.1, it's >>>>>>>>>>> working with rofs, but still having the same issue when using zfs, >>>>>>>>>>> even >>>>>>>>>>> after I correct the path for libz. >>>>>>>>>>> >>>>>>>>>>> On Saturday, December 5, 2020 at 5:18:37 PM UTC-7 Matthew >>>>>>>>>>> Kenigsberg wrote: >>>>>>>>>>> >>>>>>>>>>>> gcc version 9.3.0 (GCC) >>>>>>>>>>>> QEMU emulator version 5.1.0 >>>>>>>>>>>> >>>>>>>>>>>> Running with fs=rofs I get the error: >>>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 369, >>>>>>>>>>>> in <module> >>>>>>>>>>>> main() >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 366, >>>>>>>>>>>> in main >>>>>>>>>>>> gen_image(outfile, manifest) >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 269, >>>>>>>>>>>> in gen_image >>>>>>>>>>>> system_structure_block, bytes_written = write_fs(fp, >>>>>>>>>>>> manifest) >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 246, >>>>>>>>>>>> in write_fs >>>>>>>>>>>> count, directory_entries_index = write_dir(fp, >>>>>>>>>>>> manifest.get(''), '', manifest) >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 207, >>>>>>>>>>>> in write_dir >>>>>>>>>>>> count, directory_entries_index = write_dir(fp, val, dirpath >>>>>>>>>>>> + '/' + entry, manifest) >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 207, >>>>>>>>>>>> in write_dir >>>>>>>>>>>> count, directory_entries_index = write_dir(fp, val, dirpath >>>>>>>>>>>> + '/' + entry, manifest) >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 222, >>>>>>>>>>>> in write_dir >>>>>>>>>>>> inode.count = write_file(fp, val) >>>>>>>>>>>> File "/home/matthew/osv/scripts/gen-rofs-img.py", line 164, >>>>>>>>>>>> in write_file >>>>>>>>>>>> with open(path, 'rb') as f: >>>>>>>>>>>> FileNotFoundError: [Errno 2] No such file or directory: >>>>>>>>>>>> 'libz.so.1' >>>>>>>>>>>> >>>>>>>>>>>> I think that's from this line in usr.manifest? >>>>>>>>>>>> /usr/lib/libz.so.1: libz.so.1 >>>>>>>>>>>> >>>>>>>>>>>> Don't have zlib in the manifest without fs=rofs, and I think >>>>>>>>>>>> zpool uses it? >>>>>>>>>>>> >>>>>>>>>>>> Looking into it... >>>>>>>>>>>> On Saturday, December 5, 2020 at 4:36:20 PM UTC-7 >>>>>>>>>>>> [email protected] wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I can not reproduce it on Ubuntu 20.20 neither Fedora 33. Here >>>>>>>>>>>>> is the code fragment where it happens: >>>>>>>>>>>>> >>>>>>>>>>>>> 169 bool object::arch_relocate_jump_slot(symbol_module& sym, >>>>>>>>>>>>> void *addr, Elf64_Sxword addend) >>>>>>>>>>>>> >>>>>>>>>>>>> 170 { >>>>>>>>>>>>> >>>>>>>>>>>>> 171 if (sym.symbol) { >>>>>>>>>>>>> >>>>>>>>>>>>> 172 *static_cast<void**>(addr) = sym.relocated_addr(); >>>>>>>>>>>>> >>>>>>>>>>>>> 173 return true; >>>>>>>>>>>>> >>>>>>>>>>>>> 174 } else { >>>>>>>>>>>>> >>>>>>>>>>>>> 175 return false; >>>>>>>>>>>>> >>>>>>>>>>>>> 176 } >>>>>>>>>>>>> >>>>>>>>>>>>> 177 } >>>>>>>>>>>>> It looks like writing at the addr 0x100000040ca8 in line 172 >>>>>>>>>>>>> caused the fault. Why? >>>>>>>>>>>>> >>>>>>>>>>>>> And then the 2nd page fault in the gdb backtrace as the 1st >>>>>>>>>>>>> one was being handled (not sure if that is a bug or just a state >>>>>>>>>>>>> of loading >>>>>>>>>>>>> of a program). >>>>>>>>>>>>> >>>>>>>>>>>>> 981 if (dynamic_exists(DT_HASH)) { >>>>>>>>>>>>> >>>>>>>>>>>>> 982 auto hashtab = dynamic_ptr<Elf64_Word>(DT_HASH); >>>>>>>>>>>>> >>>>>>>>>>>>> 983 return hashtab[1]; >>>>>>>>>>>>> >>>>>>>>>>>>> 984 } >>>>>>>>>>>>> Is something wrong with the elf files cpiod.so, mkfs.so or >>>>>>>>>>>>> zfs.so or something? >>>>>>>>>>>>> >>>>>>>>>>>>> Can you try to do the same with ROFS? >>>>>>>>>>>>> >>>>>>>>>>>>> fs=rofs >>>>>>>>>>>>> On Saturday, December 5, 2020 at 5:44:12 PM UTC-5 Matthew >>>>>>>>>>>>> Kenigsberg wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Struggling to get scripts/build to run on NixOS because I'm >>>>>>>>>>>>>> getting a page fault. NixOS does keep shared libraries in >>>>>>>>>>>>>> nonstandard >>>>>>>>>>>>>> locations, not sure if that's breaking something. More details >>>>>>>>>>>>>> below, but >>>>>>>>>>>>>> any ideas? >>>>>>>>>>>>>> >>>>>>>>>>>>>> As far as I can tell, the error is caused by >>>>>>>>>>>>>> tools/mkfs/mkfs.cc:71: >>>>>>>>>>>>>> run_cmd("/zpool.so", zpool_args); >>>>>>>>>>>>>> >>>>>>>>>>>>>> The error from scripts/build: >>>>>>>>>>>>>> >>>>>>>>>>>>>> OSv v0.55.0-145-g97f17a7a >>>>>>>>>>>>>> eth0: 192.168.122.15 >>>>>>>>>>>>>> Booted up in 154.38 ms >>>>>>>>>>>>>> Cmdline: /tools/mkfs.so; /tools/cpiod.so --prefix /zfs/zfs/; >>>>>>>>>>>>>> /zfs.so set compression=off osv >>>>>>>>>>>>>> Running mkfs... >>>>>>>>>>>>>> page fault outside application, addr: 0x0000100000040ca8 >>>>>>>>>>>>>> [registers] >>>>>>>>>>>>>> RIP: 0x000000004039c25a >>>>>>>>>>>>>> <elf::object::arch_relocate_jump_slot(elf::symbol_module&, >>>>>>>>>>>>>> void*, long)+26> >>>>>>>>>>>>>> RFL: 0x0000000000010202 CS: 0x0000000000000008 SS: >>>>>>>>>>>>>> 0x0000000000000010 >>>>>>>>>>>>>> RAX: 0x000010000007a340 RBX: 0x0000100000040ca8 RCX: >>>>>>>>>>>>>> 0x000010000006abb0 RDX: 0x0000000000000002 >>>>>>>>>>>>>> RSI: 0x00002000001f6f70 RDI: 0xffffa00001058c00 RBP: >>>>>>>>>>>>>> 0x00002000001f6f30 R8: 0xffffa00000a68460 >>>>>>>>>>>>>> R9: 0xffffa00000f18da0 R10: 0x0000000000000000 R11: >>>>>>>>>>>>>> 0x00000000409dd380 R12: 0xffffa00000f18c00 >>>>>>>>>>>>>> R13: 0xffffa00000f18da0 R14: 0x0000000000000000 R15: >>>>>>>>>>>>>> 0x00000000409dd380 RSP: 0x00002000001f6f20 >>>>>>>>>>>>>> Aborted >>>>>>>>>>>>>> >>>>>>>>>>>>>> [backtrace] >>>>>>>>>>>>>> 0x00000000403458d3 <???+1077172435> >>>>>>>>>>>>>> 0x00000000403477ce <mmu::vm_fault(unsigned long, >>>>>>>>>>>>>> exception_frame*)+350> >>>>>>>>>>>>>> 0x0000000040398ba2 <page_fault+162> >>>>>>>>>>>>>> 0x0000000040397a16 <???+1077508630> >>>>>>>>>>>>>> 0x0000000040360a13 <elf::object::resolve_pltgot(unsigned >>>>>>>>>>>>>> int)+387> >>>>>>>>>>>>>> 0x0000000040360c38 <elf_resolve_pltgot+56> >>>>>>>>>>>>>> 0x000000004039764f <???+1077507663> >>>>>>>>>>>>>> 0xffffa000012b880f <???+19630095> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Trying to get a backtrace after connecting with gdb: >>>>>>>>>>>>>> (gdb) bt >>>>>>>>>>>>>> #0 abort (fmt=fmt@entry=0x40644b90 "Assertion failed: %s >>>>>>>>>>>>>> (%s: %s: %d)\n") at runtime.cc:105 >>>>>>>>>>>>>> #1 0x000000004023c6fb in __assert_fail >>>>>>>>>>>>>> (expr=expr@entry=0x40672cf8 "ef->rflags & processor::rflags_if", >>>>>>>>>>>>>> file=file@entry=0x40672d25 "arch/x64/mmu.cc", >>>>>>>>>>>>>> line=line@entry=38, func=func@entry=0x40672d1a "page_fault") >>>>>>>>>>>>>> at runtime.cc:139 >>>>>>>>>>>>>> #2 0x0000000040398c05 in page_fault (ef=0xffff800000015048) >>>>>>>>>>>>>> at arch/x64/arch-cpu.hh:107 >>>>>>>>>>>>>> #3 <signal handler called> >>>>>>>>>>>>>> #4 0x000000004035c879 in elf::object::symtab_len >>>>>>>>>>>>>> (this=0xffffa00000f18c00) at core/elf.cc:983 >>>>>>>>>>>>>> #5 0x000000004035c938 in elf::object::lookup_addr >>>>>>>>>>>>>> (this=0xffffa00000f18c00, addr=addr@entry=0x1000000254ce) >>>>>>>>>>>>>> at core/elf.cc:1015 >>>>>>>>>>>>>> #6 0x000000004035cb07 in elf::program::<lambda(const >>>>>>>>>>>>>> elf::program::modules_list&)>::operator() ( >>>>>>>>>>>>>> __closure=<synthetic pointer>, __closure=<synthetic >>>>>>>>>>>>>> pointer>, ml=...) at core/elf.cc:1620 >>>>>>>>>>>>>> #7 elf::program::with_modules<elf::program::lookup_addr(void >>>>>>>>>>>>>> const*)::<lambda(const elf::program::modules_list&)> > >>>>>>>>>>>>>> (f=..., this=0xffffa00000097e70) at include/osv/elf.hh:702 >>>>>>>>>>>>>> #8 elf::program::lookup_addr (this=0xffffa00000097e70, >>>>>>>>>>>>>> addr=addr@entry=0x1000000254ce) at core/elf.cc:1617 >>>>>>>>>>>>>> #9 0x00000000404357cc in osv::lookup_name_demangled >>>>>>>>>>>>>> (addr=addr@entry=0x1000000254ce, >>>>>>>>>>>>>> buf=buf@entry=0xffff8000012146d0 "???+19630095", >>>>>>>>>>>>>> len=len@entry=1024) at core/demangle.cc:47 >>>>>>>>>>>>>> #10 0x000000004023c4e0 in print_backtrace () at runtime.cc:85 >>>>>>>>>>>>>> #11 0x000000004023c6b4 in abort (fmt=fmt@entry=0x40644a9f >>>>>>>>>>>>>> "Aborted\n") at runtime.cc:121 >>>>>>>>>>>>>> #12 0x0000000040202989 in abort () at runtime.cc:98 >>>>>>>>>>>>>> #13 0x00000000403458d4 in mmu::vm_sigsegv >>>>>>>>>>>>>> (ef=0xffff800001215068, addr=<optimized out>) at core/mmu.cc:1314 >>>>>>>>>>>>>> #14 mmu::vm_sigsegv (addr=<optimized out>, >>>>>>>>>>>>>> ef=0xffff800001215068) at core/mmu.cc:1308 >>>>>>>>>>>>>> #15 0x00000000403477cf in mmu::vm_fault >>>>>>>>>>>>>> (addr=addr@entry=17592186309800, ef=ef@entry=0xffff800001215068) >>>>>>>>>>>>>> at core/mmu.cc:1328 >>>>>>>>>>>>>> #16 0x0000000040398ba3 in page_fault (ef=0xffff800001215068) >>>>>>>>>>>>>> at arch/x64/mmu.cc:42 >>>>>>>>>>>>>> #17 <signal handler called> >>>>>>>>>>>>>> #18 0x000000004039c25a in >>>>>>>>>>>>>> elf::object::arch_relocate_jump_slot >>>>>>>>>>>>>> (this=this@entry=0xffffa00000f18c00, >>>>>>>>>>>>>> sym=..., >>>>>>>>>>>>>> addr=addr@entry=0x100000040ca8, addend=addend@entry=0) at >>>>>>>>>>>>>> arch/x64/arch-elf.cc:172 >>>>>>>>>>>>>> #19 0x0000000040360a14 in elf::object::resolve_pltgot >>>>>>>>>>>>>> (this=0xffffa00000f18c00, index=<optimized out>) >>>>>>>>>>>>>> at core/elf.cc:843 >>>>>>>>>>>>>> #20 0x0000000040360c39 in elf_resolve_pltgot (index=308, >>>>>>>>>>>>>> obj=0xffffa00000f18c00) at core/elf.cc:1860 >>>>>>>>>>>>>> #21 0x0000000040397650 in __elf_resolve_pltgot () at >>>>>>>>>>>>>> arch/x64/elf-dl.S:47 >>>>>>>>>>>>>> #22 0x00001000000254cf in ?? () >>>>>>>>>>>>>> #23 0xffffa000012b8800 in ?? () >>>>>>>>>>>>>> #24 0x00002000001f74a0 in ?? () >>>>>>>>>>>>>> #25 0x00001000000254cf in ?? () >>>>>>>>>>>>>> #26 0x00002000001f7480 in ?? () >>>>>>>>>>>>>> #27 0x00000000403f241c in calloc (nmemb=<optimized out>, >>>>>>>>>>>>>> size=<optimized out>) at core/mempool.cc:1811 >>>>>>>>>>>>>> #28 0xffff900000a98000 in ?? () >>>>>>>>>>>>>> #29 0x0000000000000000 in ?? () >>>>>>>>>>>>>> On Saturday, November 28, 2020 at 1:39:46 PM UTC-7 Matthew >>>>>>>>>>>>>> Kenigsberg wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'll send something, might take a bit before I find time to >>>>>>>>>>>>>>> work on it though. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Matthew >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Saturday, November 28, 2020 at 1:11:11 PM UTC-7 Roman >>>>>>>>>>>>>>> Shaposhnik wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Nov 24, 2020 at 8:03 AM Waldek Kozaczuk < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > Hey, >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > Send a patch with a new app that could demonstrate it, >>>>>>>>>>>>>>>> please, if you can. I would like to see it. Sounds like a nice >>>>>>>>>>>>>>>> improvement. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> FWIW: I'd love to see it too -- been meaning to play with >>>>>>>>>>>>>>>> Nix and this >>>>>>>>>>>>>>>> gives me a perfect excuse ;-) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Roman. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>>> in the Google Groups "OSv Development" group. >>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>> https://groups.google.com/d/topic/osv-dev/rhjHPr7OBEw/unsubscribe >>>>>>>>>>> . >>>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>>> to [email protected]. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/osv-dev/7913b79b-6c06-4f2a-95d3-9dc44e45eb45n%40googlegroups.com >>>>>>>>>>> >>>>>>>>>>> <https://groups.google.com/d/msgid/osv-dev/7913b79b-6c06-4f2a-95d3-9dc44e45eb45n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/f0ff8180-ff7f-4464-8499-b8709aec480an%40googlegroups.com.
