bug#58033: A bug in file-dynamic-info used by validate-runpath in gnu-build-system and others.
It appears I found a bug in guix triggered by certain binary data
present in an ELF header.
While running a validate runpath phase of a build-system for a new
package that is not a part of guix I encountered unusual errors
reported for certain binary files. Those binary files RUNPATHs were
modified by patchelf by adding a colon character followed by a
/gnu/store directory folder to runpath. As far as readelf/patchelf and
ld are concerned the binary files appear to contain a valid elf header
with valid DT_DYNAMIC section.
However, something in the ELF header triggers a bug in Guix's
file-runpath / file-dynamic-info procedure that result in the
following output:
scheme@(guix-user)> (file-dynamic-info
"/gnu/store/20z595j5jas5ri3nrza5465gbxwf9kmf-python-redacted/lib/python3.9/site-packages/torch/bin/FileStoreTest")
$13 = #< soname: #f needed: ("" "" "" "" "" "" "" ""
"" "" "" "" "" "" "") rpath: () runpath: ()>
scheme@(guix-user)>
As can be seen above file-dynamic-info is unable to read the NEEDED
items, but correctly reports 8 of them. Additionally it is unable to
read the RUNPATH variable and returns an empty string. All those
values are populated in the header as shown below.
readelf -d reports the following correct DT_DYNAMIC:
[luk@archczop guix]$ readelf -d
/gnu/store/20z595j5jas5ri3nrza5465gbxwf9kmf-python-redacted/lib/python3.9/site-packages/torch/bin/FileStoreTest
Dynamic section at offset 0xfcf0 contains 40 entries:
TagType Name/Value
0x0001 (NEEDED) Shared library: [libtorch_cpu.so]
0x0001 (NEEDED) Shared library:
[libgtest_main.so.1.11.0]
0x0001 (NEEDED) Shared library: [libgtest.so.1.11.0]
0x0001 (NEEDED) Shared library: [libpthread.so.0]
0x0001 (NEEDED) Shared library: [libprotobuf.so.28]
0x0001 (NEEDED) Shared library: [libc10.so]
0x0001 (NEEDED) Shared library:
[libmkl_intel_lp64.so.2]
0x0001 (NEEDED) Shared library:
[libmkl_gnu_thread.so.2]
0x0001 (NEEDED) Shared library: [libmkl_core.so.2]
0x0001 (NEEDED) Shared library: [libdl.so.2]
0x0001 (NEEDED) Shared library: [libstdc++.so.6]
0x0001 (NEEDED) Shared library: [libm.so.6]
0x0001 (NEEDED) Shared library: [libgomp.so.1]
0x0001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0001 (NEEDED) Shared library: [libc.so.6]
0x001d (RUNPATH)Library runpath:
[$ORIGIN/../lib:/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib:/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib:/gnu/store/mbzav28sik3zr3kbw1jyh4qk3zmkh6xn-googletest-1.11.0/lib:/gnu/store/9pyydl5w9xnz1qm56sxn1zh4qny6fkxz-protobuf-3.17.3/lib:/gnu/store/fj5npv9kpsiihrzpzhdlcz5q6bax15s8-mkl-2022.1.0/lib:/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/10.3.0/../../..:/gnu/store/janq8zcngwc7120gyj41cc2yysk7p9i5-nvidia-libs-515.65.01/lib]
0x000c (INIT) 0x405000
0x000d (FINI) 0x40bed4
0x0019 (INIT_ARRAY) 0x40f6c8
0x001b (INIT_ARRAYSZ) 16 (bytes)
0x001a (FINI_ARRAY) 0x40f6d8
0x001c (FINI_ARRAYSZ) 8 (bytes)
0x0004 (HASH) 0x402210
0x6ef5 (GNU_HASH) 0x401fa8
0x0005 (STRTAB) 0x3ff350
0x0006 (SYMTAB) 0x4011b0
0x000a (STRSZ) 7769 (bytes)
0x000b (SYMENT) 24 (bytes)
0x0015 (DEBUG) 0x0
0x0003 (PLTGOT) 0x41
0x0002 (PLTRELSZ) 1680 (bytes)
0x0014 (PLTREL) RELA
0x0017 (JMPREL) 0x4039e0
0x0007 (RELA) 0x403710
0x0008 (RELASZ) 720 (bytes)
0x0009 (RELAENT)24 (bytes)
0x6ffe (VERNEED)0x403620
0x6fff (VERNEEDNUM) 4
0x6ff0 (VERSYM) 0x4034f0
0x (NULL) 0x0
Therefore deeper analysis of what is in the binary header that
triggers the bug is required. I have a number of those binary files. I
attached the smallest one(74kB) base64 encoded to this email. It is
the binary named FileStoreTest used in the report above. If it gets
stripped from the message I'll reply by submitting it in the body of
the message.
FileStoreTest.b64
Description: Binary data
bug#58033: A bug in file-dynamic-info used by validate-runpath in gnu-build-system and others.
After further troubleshooting it appears the elf binary might be malformed. I've tried to run the executable on another machine and ld complained like this: $ ./FileStoreTest ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: ./FileStoreTest: no version information available (required by ./FileStoreTest) ./FileStoreTest: symbol lookup error: ./FileStoreTest: undefined symbol: , version Further testing is needed to tell if this is indeed a bug in guix not processing a correct header, or simply failing to display a meaningful error message. Regards, Lukasz
bug#58033: A bug in file-dynamic-info used by validate-runpath in gnu-build-system and others.
Also, to ensure all the information is provided. This is the code that resulted in the binary header being transformed: (add-after 'install 'fix-issue-with-libs (lambda* (#:key inputs outputs #:allow-other-keys) (chdir "..") (use-modules (ice-9 ftw) (ice-9 regex) (ice-9 rdelim) (ice-9 popen) (ice-9 textual-ports)) (let* ((libdir (string-append #$output "/lib"))) ;; -- ;; patchelf (define (get-rpaths file) (format #t "Getting rpaths from ~a ...~%" file) (let* ((port (open-input-pipe (string-append "patchelf --print-rpath " file))) (str (read-line port))) ; from (ice-9 rdelim) (close-pipe port) str)) (define (patch-elf file) (format #t "Patching ~a ...~%" file) (define rpath (string-append (get-rpaths file) ":" #$extra-libs "/lib")) (display (string-append "We're setting rpath:" rpath)) (invoke "patchelf" "--set-rpath" rpath file)) (for-each (lambda (file) (when (elf-file? file) (patch-elf file))) (find-files #$output ".*") I can run the patch-elf procedure in repl and it runs fine, but being run during the build it results in the problematic elf header. The build was run twice with the same result both times. Regards, Lukasz
bug#58033: A bug in file-dynamic-info used by validate-runpath in gnu-build-system and others.
Sorry, for multiple emails with the same content. There was a delay of over 10h in those emails showing up and I thought they were not getting through. Regarding the issue. It seems it has to do with the strip phase. I managed to replicate the whole issue outside the build system by taking binaries backed up after install. Then if I run $ strip --strip-unneeded --enable-deterministic-archives file the files can be run fine, but if I use patchelf to add an extra folder to the rpath strip complains like this: $ strip --strip-unneeded --enable-deterministic-archives /home/luk/dev/backup_FileStoreTest strip: /home/luk/dev/stt5WKN1: warning: allocated section `.dynstr' not in segment Then the binary has its elf header mangled as described previously. By copying the unmodified file, modifying rpath and running strip a couple of times I found that there is no problem if the rpath change results in rpath of the same or shorter length, but adding even one byte to it makes 'strip' later complain and mangle the binary. Regards, Lukasz
bug#58033: A bug in file-dynamic-info used by validate-runpath in gnu-build-system and others.
Hi Lukasz, Lukasz Olszewski skribis: > Then if I run $ strip --strip-unneeded --enable-deterministic-archives > file the files can be run fine, but if I use patchelf to add an extra > folder to the rpath strip complains like this: > $ strip --strip-unneeded --enable-deterministic-archives > /home/luk/dev/backup_FileStoreTest > strip: /home/luk/dev/stt5WKN1: warning: allocated section `.dynstr' > not in segment > > Then the binary has its elf header mangled as described previously. > > By copying the unmodified file, modifying rpath and running strip a > couple of times I found that there is no problem if the rpath change > results in rpath of the same or shorter length, but adding even one > byte to it makes 'strip' later complain and mangle the binary. I believe PatchELF has the potential to break binaries, especially when trying to extend RUNPATH (the data has to fit in the string table; PatchELF is supposed to be able to grow the string table as needed, but there might be bugs.) It looks like a workaround is to not run ‘strip’, right? Ludo’.
