Have you guys considered running elf fuzzing: https://github.com/IOActive/Melkor_ELF_Fuzzer? This tool pops up when I'm searching for elf standard testing. It doesn't test TLS handing though.
On Mon, Dec 16, 2019 at 1:17 PM Waldek Kozaczuk <jwkozac...@gmail.com> wrote: > Regarding local STT_TLS based on this - > https://www.openwall.com/lists/musl/2019/05/24/1 - here is the simple > reproducer: > > cat vers > { > global: > get_tls_var; > local: > *; > }; > > cat dso.c > __thread int tlsvar = 11; > int get_tls_var(void) { > return tlsvar; > } > > cat main.c > #include <stdio.h> > int get_tls_var(void); > int main() { > printf("get_tls_var(): %d\n", get_tls_var()); > return 0; > } > > > So now If I build those two like this: > > gcc dso.c -fpic -shared -Wl,-version-script=vers -o libdso.so > gcc main.c libdso.so -Wl,-rpath,'$ORIGIN' -o main > > the main program will work with or without my latest patch. > > But if we build it like that with gold linker: > gcc dso.c -fpic -shared -fuse-ld=gold -Wl,-version-script=vers -o libdso.so > gcc main.c libdso.so -Wl,-rpath,'$ORIGIN' -o main > > the program crashes like so: > /usr/lib/libdso.so: failed looking up symbol tlsvar > > [backtrace] > 0x0000000040364155 <elf::object::symbol(unsigned int, bool)+1637> > 0x00000000403b28b9 <elf::object::arch_relocate_rela(unsigned int, unsigned > int, void*, long)+649> > 0x000000004035fcb3 <elf::object::relocate_rela()+211> > 0x000000004036129a <elf::object::relocate()+250> > 0x0000000040365a20 > <elf::program::load_object(std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> >, > std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > > >, > std::vector<std::shared_ptr<elf::object>, > std::allocator<std::shared_ptr<elf::object> > >&)+2416> > 0x0000000040364a69 > <elf::object::load_needed(std::vector<std::shared_ptr<elf::object>, > std::allocator<std::shared_ptr<elf::object> > >&)+1209> > 0x0000000040365a18 > <elf::program::load_object(std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> >, > std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > > >, > std::vector<std::shared_ptr<elf::object>, > std::allocator<std::shared_ptr<elf::object> > >&)+2408> > 0x0000000040366a79 > <elf::program::get_library(std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> >, > std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > > >, bool)+345> > 0x0000000040443882 > <osv::application::application(std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > const&, > std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > > > const&, bool, > std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >, std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> >, > std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > >, > std::allocator<std::pair<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > const, > std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > > > > const*, std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > const&, > std::function<0x0000000040443bd2 > <osv::application::run(std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > const&, > std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > > > const&, bool, > std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >, std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> >, > std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > >, > std::allocator<std::pair<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > const, > std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > > > > const*, std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > const&, std::function<void > ()>0x0000000040443e20 > <osv::application::run(std::vector<std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> >, > std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > > > const&)+80> > 0x000000004022d6ee <do_main_thread(void*)+3230> > 0x00000000404782b9 <???+1078428345> > 0x000000004040f29b <thread_main_c+43> > 0x00000000403ae8b2 <???+1077602482> > > and works the latest patch correctly. > > On Monday, December 16, 2019 at 12:01:17 PM UTC-5, Waldek Kozaczuk wrote: >> >> It could be that musl also does not handle LOCAL STT_TLS well when ELF >> linked with gold linker (and glibc does). Here is a thread I have found >> that seems to be applicable to us as well - >> https://www.openwall.com/lists/musl/2019/05/25/2. >> >> This pseudo-patch (includes yours so you might need to apply manually) >> seems to handle this: >> >> diff --git a/core/elf.cc b/core/elf.cc >> index 13f0c87a..de262e35 100644 >> --- a/core/elf.cc >> +++ b/core/elf.cc >> @@ -642,10 +642,17 @@ symbol_module object::symbol(unsigned idx, bool >> ignore_missing) >> auto symtab = dynamic_ptr<Elf64_Sym>(DT_SYMTAB); >> assert(dynamic_val(DT_SYMENT) == sizeof(Elf64_Sym)); >> auto sym = &symtab[idx]; >> + auto type = sym->st_info & 15; >> + if (type == STT_SECTION) { >> + return symbol_module(sym, this); >> + } >> + auto binding = symbol_binding(*sym); >> + if (type == STT_TLS && binding == STB_LOCAL) { >> + return symbol_module(sym, this); >> + } >> auto nameidx = sym->st_name; >> auto name = dynamic_ptr<const char>(DT_STRTAB) + nameidx; >> auto ret = _prog.lookup(name); >> - auto binding = symbol_binding(*sym); >> if (!ret.symbol && binding == STB_WEAK) { >> return symbol_module(sym, this); >> } >> >> It seems to do the right thing but I am not sure it is fully correct. I >> think it will also fix the other similar issue I found when trying to run >> dotnet and failing to resolve LOCAL TLS symbol as well (back there it was >> STV_HIDDEN, here os STV_DEFAULT but this aspect does not matter I think). >> >> After that I hit clock_gettime issue which can be addressed like this >> (again not really sure if this patch is correct): >> diff --git a/libc/time.cc b/libc/time.cc >> index 8aa11580..9b23a18f 100644 >> --- a/libc/time.cc >> +++ b/libc/time.cc >> @@ -60,6 +60,8 @@ int clock_gettime(clockid_t clk_id, struct timespec* ts) >> switch (clk_id) { >> case CLOCK_BOOTTIME: >> case CLOCK_MONOTONIC: >> + case CLOCK_MONOTONIC_COARSE: >> + case CLOCK_MONOTONIC_RAW: >> fill_ts(osv::clock::uptime::now().time_since_epoch(), ts); >> break; >> case CLOCK_REALTIME: >> >> Waldek >> >> On Sunday, December 15, 2019 at 8:40:05 PM UTC-5, Waldek Kozaczuk wrote: >>> >>> Copying to the list. >>> >>> On Sun, Dec 15, 2019 at 20:38 Waldek Kozaczuk <jwkozac...@gmail.com> >>> wrote: >>> >>>> It might be similar (or have same root cause) as the issue with >>>> gCurrentThreadInfo reported here - >>>> https://groups.google.com/forum/m/#!topic/osv-dev/k69cHw7qvTg. In both >>>> cases the symbols are TLS but in other case it was a HIDDEN symbol in this >>>> one it is DEFAULT. There maybe be some sort of bug in looking up TLS >>>> symbols. This one is also LOCAL (I do not remember the other case). I >>>> wonder if there is similar rule like with STT_SECTION - if TLS and LOCAL >>>> the symbol_module(sym, this). I wonder what musl is doing in this case. >>>> >>>> On Sun, Dec 15, 2019 at 20:29 zhiting zhu <zhiti...@cs.utexas.edu> >>>> wrote: >>>> >>>>> I comment out version_symtab related code and it's still having the >>>>> same problem. >>>>> >>>>> It's still failing hash comparison. Maybe I miss some other places? >>>>> The hash comparison happens before version checking. >>>>> >>>>> Best, >>>>> Zhiting >>>>> >>>>> >>>>> >>>>> >>>>> On Sun, Dec 15, 2019 at 6:31 PM Waldek Kozaczuk <jwkozac...@gmail.com> >>>>> wrote: >>>>> >>>>>> Try to disable the version check - it might be the other bug when elf >>>>>> is referencing its own old symbol. >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>> On Dec 15, 2019, at 19:09, zhiting zhu <zhiti...@cs.utexas.edu> >>>>>> wrote: >>>>>> >>>>>> >>>>>> I encounter another failed to lookup symbol error. It's >>>>>> >>>>>> 2813: 0000000000000058 8 TLS LOCAL DEFAULT 20 >>>>>> _ZN9grpc_core7ExecCtx9exe >>>>>> >>>>>> The symbol is defined in >>>>>> tensorflow/python/_pywrap_tensorflow_internal.so. I check symbol >>>>>> visibility. It's visible. I add some print out in lookup_symbol_gnu. It >>>>>> seems it can't find the symbol in that so despite it exists. I look at >>>>>> the >>>>>> checks and all the hash comparison failed (the following line). >>>>>> >>>>>> if ((chains[idx] & ~1) != (hashval & ~1)) { >>>>>> >>>>>> I'm trying to compare with the musl code: >>>>>> >>>>>> https://github.com/ifduyue/musl/blob/79f653c6bc2881dd6855299c908a442f56cb7c2b/ldso/dynlink.c#L230-L249 >>>>>> >>>>>> I don't find that much of a difference. I check the hash calculation. >>>>>> gnu_hash in musl and dl_new_hash outputs the same number for that symbol >>>>>> name. >>>>>> >>>>>> Best, >>>>>> Zhiting >>>>>> >>>>>> On Sun, Dec 15, 2019 at 12:51 PM zhiting zhu <zhiti...@cs.utexas.edu> >>>>>> wrote: >>>>>> >>>>>>> Thanks for the explanation. >>>>>>> >>>>>>> This patch seems does the job: >>>>>>> >>>>>>> Author: Zhiting Zhu <zhiti...@cs.utexas.edu> >>>>>>> Date: Sun Dec 15 00:57:36 2019 -0600 >>>>>>> >>>>>>> handling STT_SECTION symbol >>>>>>> >>>>>>> Signed-off-by: Zhiting Zhu <zhiti...@cs.utexas.edu> >>>>>>> >>>>>>> diff --git a/core/elf.cc b/core/elf.cc >>>>>>> index e2b5b667..a5be192b 100644 >>>>>>> --- a/core/elf.cc >>>>>>> +++ b/core/elf.cc >>>>>>> @@ -641,6 +641,10 @@ symbol_module object::symbol(unsigned idx, bool >>>>>>> ignore_missing) >>>>>>> auto symtab = dynamic_ptr<Elf64_Sym>(DT_SYMTAB); >>>>>>> assert(dynamic_val(DT_SYMENT) == sizeof(Elf64_Sym)); >>>>>>> auto sym = &symtab[idx]; >>>>>>> + auto type = sym->st_info & 15; >>>>>>> + if (type == STT_SECTION) { >>>>>>> + return symbol_module(sym, this); >>>>>>> + } >>>>>>> auto nameidx = sym->st_name; >>>>>>> auto name = dynamic_ptr<const char>(DT_STRTAB) + nameidx; >>>>>>> auto ret = _prog.lookup(name); >>>>>>> >>>>>>> For symbol visibility, would the patch you mentioned previously( >>>>>>> https://groups.google.com/forum/#!topic/osv-dev/d56plMGXi6E) fix >>>>>>> the problem? >>>>>>> >>>>>>> Best, >>>>>>> Zhiting >>>>>>> >>>>>>> On Sun, Dec 15, 2019 at 12:02 AM Waldek Kozaczuk < >>>>>>> jwkozac...@gmail.com> wrote: >>>>>>> >>>>>>>> After some trial and error, I did manage to reproduce your issue. >>>>>>>> While doing that I also got to understand other reported earlier issue >>>>>>>> with >>>>>>>> numpy and blas_memory_alloc symbol. First about it. After adding more >>>>>>>> debug >>>>>>>> here is what I saw and it makes all self-explanatory: >>>>>>>> >>>>>>>> ELF [tid:253, /lib/python3.7/site-packages/numpy/.libs/ >>>>>>>> libopenblasp-r0-34a18dc3.3.7.so]: setprivate set to: 1 >>>>>>>> ELF [tid:253, /lib/python3.7/site-packages/numpy/.libs/ >>>>>>>> libopenblasp-r0-34a18dc3.3.7.so]: Executing DT_INIT function >>>>>>>> ELF [tid:253, /lib/python3.7/site-packages/numpy/.libs/ >>>>>>>> libopenblasp-r0-34a18dc3.3.7.so]: Finished executing DT_INIT >>>>>>>> function >>>>>>>> ELF [tid:253, /lib/python3.7/site-packages/numpy/.libs/ >>>>>>>> libopenblasp-r0-34a18dc3.3.7.so]: Executing 2 DT_INIT_ARRAYSZ >>>>>>>> functions >>>>>>>> ELF [tid:254, /libvdso.so]: Not found in /libvdso.so! >>>>>>>> ELF [tid:254, /usr/lib/libexpat.so.1]: Not found in >>>>>>>> /usr/lib/libexpat.so.1! >>>>>>>> ELF [tid:254, /usr/lib/libz.so.1]: Not found in /usr/lib/libz.so.1! >>>>>>>> ELF [tid:254, /lib/python3.7/lib-dynload/ >>>>>>>> readline.cpython-37m-x86_64-linux-gnu.so]: Not found in >>>>>>>> /lib/python3.7/lib-dynload/readline.cpython-37m-x86_64-linux-gnu.so >>>>>>>> ! >>>>>>>> ELF [tid:254, /usr/lib/libtinfo.so.6]: Not found in >>>>>>>> /usr/lib/libtinfo.so.6! >>>>>>>> ELF [tid:254, /lib/python3.7/site-packages/numpy/core/_ >>>>>>>> multiarray_umath.cpython-37m-x86_64-linux-gnu.so]: >>>>>>>> blas_memory_alloc _visibility: 253, visible: 0! >>>>>>>> ELF [tid:254, /lib/python3.7/site-packages/numpy/.libs/ >>>>>>>> libopenblasp-r0-34a18dc3.3.7.so]: blas_memory_alloc _visibility: >>>>>>>> 253, visible: 0! >>>>>>>> ELF [tid:254, ]: Not found in ! >>>>>>>> /lib/python3.7/site-packages/numpy/.libs/ >>>>>>>> libopenblasp-r0-34a18dc3.3.7.so: failed looking up symbol >>>>>>>> blas_memory_alloc >>>>>>>> >>>>>>>> [backtrace] >>>>>>>> 0x0000000040364205 <elf::object::symbol(unsigned int, bool)+1637> >>>>>>>> 0x0000000040364306 <elf::object::resolve_pltgot(unsigned int)+134> >>>>>>>> 0x000000004036462c <elf_resolve_pltgot+60> >>>>>>>> 0x00000000403ad61f <???+1077597727> >>>>>>>> 0x0000200000780f6f <???+7868271> >>>>>>>> 0x0000000000000003 <???+3> >>>>>>>> >>>>>>>> As you can see the symbol blas_memory_alloc is being looked up by >>>>>>>> one of DT_INIT_ARRAYSZ functions but in another thread - 254 - >>>>>>>> than when the function was actually called. Our "private/visible" >>>>>>>> mechanism >>>>>>>> hides this symbol from other threads. We probably should check if the >>>>>>>> object is looking for its own symbol or fix it in another way. >>>>>>>> >>>>>>>> Your newest problem I think is related to the fact that this symbol >>>>>>>> is STT_SECTION and we are not handling it right. Here is what I found >>>>>>>> in >>>>>>>> one of the blogs: >>>>>>>> >>>>>>>> "*STT_SECTION*When I first started learning about ELF, and someone >>>>>>>> would say something about "section symbols", I thought they meant a >>>>>>>> symbol >>>>>>>> from some given section. That's not it though: A section symbol is a >>>>>>>> symbol >>>>>>>> that is used to refer to the section itself. They are used mainly when >>>>>>>> performing relocations, which are often specified in the form of >>>>>>>> "modify >>>>>>>> the value at offset XXX relative to the start of section YYY"." >>>>>>>> >>>>>>>> Here is related code in musl dynamic linker I think - >>>>>>>> https://github.com/ifduyue/musl/blob/79f653c6bc2881dd6855299c908a442f56cb7c2b/ldso/dynlink.c#L346-L371 >>>>>>>> (look >>>>>>>> for STT_SECTION if). >>>>>>>> >>>>>>>> I think we should detect if the relocated symbol is of type >>>>>>>> STT_SECTION and change the handling of R_X86_64_DTPMOD64 and NOT >>>>>>>> lookup the >>>>>>>> symbol in other modules but instead use the module index of the object >>>>>>>> whose symbol is being relocated. >>>>>>>> >>>>>>>> On Saturday, December 14, 2019 at 10:46:36 AM UTC-5, zhiting zhu >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I think the following would work. >>>>>>>>> >>>>>>>>> python3 -m pip install tensorflow==1.14 (I think google's official >>>>>>>>> binary should work on your machine. I check that they have the same >>>>>>>>> problem). >>>>>>>>> >>>>>>>>> Replace the GET file in apps/python3x with the one attached. >>>>>>>>> >>>>>>>>> ./scripts/build fs_size_mb=1024 image=python3x >>>>>>>>> >>>>>>>>> ./scripts/run.py -r >>>>>>>>> >>>>>>>>> In the python cli, type "import tensorflow". >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Zhiting >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Dec 14, 2019 at 8:23 AM Waldek Kozaczuk < >>>>>>>>> jwkoz...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Or dockerfile so that I can easily recreate the image. >>>>>>>>>> >>>>>>>>>> On Fri, Dec 13, 2019 at 20:31 Waldek Kozaczuk <jwkoz...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I downloaded the wheel file but I get this error (I am using >>>>>>>>>>> Ubuntu 19.10): >>>>>>>>>>> >>>>>>>>>>> pip3 install ./tensorflow-1.14.1-cp36-cp36m-linux_x86_64.whl >>>>>>>>>>> tensorflow-1.14.1-cp36-cp36m-linux_x86_64.whl is not a supported >>>>>>>>>>> wheel on this platform. >>>>>>>>>>> >>>>>>>>>>> Could you please send us full recipe that would build the app? >>>>>>>>>>> Ideally an "app" just like any we have under osv-apps repo so that >>>>>>>>>>> I can >>>>>>>>>>> simply run: >>>>>>>>>>> ./scripts/buils image=???? >>>>>>>>>>> >>>>>>>>>>> and then run it. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Waldek >>>>>>>>>>> >>>>>>>>>>> On Friday, December 13, 2019 at 5:52:03 PM UTC-5, zhiting zhu >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> No. It's >>>>>>>>>>>> Num: Value Size Type Bind Vis Ndx Name >>>>>>>>>>>> 1: 000000000affa7f8 0 SECTION LOCAL DEFAULT 21 >>>>>>>>>>>> >>>>>>>>>>>> It's not hidden. >>>>>>>>>>>> >>>>>>>>>>>> Here's the new link: >>>>>>>>>>>> https://send.firefox.com/download/9a8bf3fa2909635f/#0ZecrR7UJwspr743vNBo6A >>>>>>>>>>>> to the file. >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Dec 13, 2019 at 4:44 PM Waldek Kozaczuk < >>>>>>>>>>>> jwkoz...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>> I am not sure but this issue is similar to what I encountered >>>>>>>>>>>>> when dealing with dotnet: >>>>>>>>>>>>> >>>>>>>>>>>>> readelf -s libcoreclr.so | grep gCurrentThreadInfo >>>>>>>>>>>>> readelf: Warning: local symbol 31 found at index >= .dynsym's >>>>>>>>>>>>> sh_info value of 1 >>>>>>>>>>>>> 31: 0000000000000000 24 TLS LOCAL HIDDEN 19 >>>>>>>>>>>>> gCurrentThreadInfo >>>>>>>>>>>>> 9799: 0000000000000000 24 TLS LOCAL HIDDEN 19 >>>>>>>>>>>>> gCurrentThreadInfo >>>>>>>>>>>>> >>>>>>>>>>>>> When you use readelf does it show it as a hidden, local TLS >>>>>>>>>>>>> symbol? >>>>>>>>>>>>> >>>>>>>>>>>>> That link with compiles expired. Can you upload it somewhere >>>>>>>>>>>>> again? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> >>>>>>>>>>>>> On Friday, December 13, 2019 at 1:16:21 PM UTC-5, zhiting zhu >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> I was reading the print wrong. >>>>>>>>>>>>>> I think the problem is this one: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 00000b23ca60 000100000010 R_X86_64_DTPMOD64 000000000aee9f58 >>>>>>>>>>>>>> .tbss + 0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The sym is 1 but in this case, .tbss is still in the same >>>>>>>>>>>>>> shared object but in arch_relocate_rela, it's going to the else >>>>>>>>>>>>>> branch that >>>>>>>>>>>>>> handles the variable is located in DIFFERENT shared object that >>>>>>>>>>>>>> the caller. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Dec 11, 2019 at 3:12 PM zhiting zhu < >>>>>>>>>>>>>> zhit...@cs.utexas.edu> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I get rid of the __VA_OPT and replace the __VA_ARGS__ with >>>>>>>>>>>>>>> ##__VA_ARGS__. ## seems to be the extension from GNU that does >>>>>>>>>>>>>>> similar >>>>>>>>>>>>>>> things. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Here's the link of the TensorFlow that I compile myself: >>>>>>>>>>>>>>> https://send.firefox.com/download/68d2a81e2cacdafb/#ui_lMMYAU9UW9e6Fd1VG7w >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Install it locally and build an osv vm that includes all the >>>>>>>>>>>>>>> dependencies: >>>>>>>>>>>>>>> "tensorflow grpc google _cffi_backend past future \ >>>>>>>>>>>>>>> absl wrapt gast astor termcolor numpy unittest libfuturize >>>>>>>>>>>>>>> \ >>>>>>>>>>>>>>> keras_applications keras_preprocessing >>>>>>>>>>>>>>> tensorflow_estimator \ >>>>>>>>>>>>>>> tensorboard". >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Then just run "import tensorflow" in python shell, it should >>>>>>>>>>>>>>> show you the error I'm looking at. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Dec 11, 2019 at 2:55 PM Waldek Kozaczuk < >>>>>>>>>>>>>>> jwkoz...@gmail.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I wonder if that is related to a similar issue as described >>>>>>>>>>>>>>>> here - >>>>>>>>>>>>>>>> https://groups.google.com/forum/#!topic/osv-dev/k69cHw7qvTg >>>>>>>>>>>>>>>> . >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will try to fix and apply this debug patch to master so >>>>>>>>>>>>>>>> it makes easier to debug it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Meanwhile, can you provide a reproducing test? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wednesday, December 11, 2019 at 3:49:07 PM UTC-5, >>>>>>>>>>>>>>>> zhiting zhu wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> After tracing inside arch_relocate_rela, it's failed in >>>>>>>>>>>>>>>>> case R_X86_64_DTPMOD64. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I print out the name index of that symbol and it's 0. I >>>>>>>>>>>>>>>>> use readelf -a to check the .rela.dyn section. >>>>>>>>>>>>>>>>> There's a line that doesn't have Sym. Value and Sym.Name + >>>>>>>>>>>>>>>>> Addend is 0. I think I'm failing at that line. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Offset Info Type Sym. Value >>>>>>>>>>>>>>>>> Sym. Name + Addend >>>>>>>>>>>>>>>>> *00000b202f90 000000000010 R_X86_64_DTPMOD64 >>>>>>>>>>>>>>>>> 0* >>>>>>>>>>>>>>>>> 00000b23ca60 000100000010 R_X86_64_DTPMOD64 >>>>>>>>>>>>>>>>> 000000000aee9f58 .tbss + 0 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Dec 11, 2019 at 10:36 AM zhiting zhu < >>>>>>>>>>>>>>>>> zhit...@cs.utexas.edu> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for the debug patch. When I apply it, g++ >>>>>>>>>>>>>>>>>> complains about "core/elf.cc:36:118: error: expected ‘)’ >>>>>>>>>>>>>>>>>> before >>>>>>>>>>>>>>>>>> ‘__VA_OPT__’" >>>>>>>>>>>>>>>>>> I think __VA_OPT__ is only added to c++2a according to >>>>>>>>>>>>>>>>>> https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html. >>>>>>>>>>>>>>>>>> Gcc 7.4.0 on Ubuntu cannot compile that. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tue, Dec 10, 2019 at 7:20 AM Waldek Kozaczuk < >>>>>>>>>>>>>>>>>> jwkoz...@gmail.com> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> You may want to try to apply this patch - >>>>>>>>>>>>>>>>>>> https://groups.google.com/forum/#!topic/osv-dev/LbnnY2Kcmak >>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>> that should provide many useful debug printouts. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> There is another patch I have sent that fixes the >>>>>>>>>>>>>>>>>>> versioned self-lookup problem - >>>>>>>>>>>>>>>>>>> https://groups.google.com/forum/#!topic/osv-dev/d56plMGXi6E. >>>>>>>>>>>>>>>>>>> I wonder if it fixes your problem (my gut tells me yours is >>>>>>>>>>>>>>>>>>> different). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Tuesday, December 10, 2019 at 3:07:09 AM UTC-5, Nadav >>>>>>>>>>>>>>>>>>> Har'El wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Mon, Dec 9, 2019 at 10:51 PM zhiting zhu < >>>>>>>>>>>>>>>>>>>> zzt...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hey, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I'm encountering this when I'm using some tensorflow >>>>>>>>>>>>>>>>>>>>> functions: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> /lib/python3.6/tensorflow/python/_pywrap_tensorflow_internal.so: >>>>>>>>>>>>>>>>>>>>> failed looking up symbol >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This is interesting, because the "failed looking up >>>>>>>>>>>>>>>>>>>> symbol" message is always followed by the name of the >>>>>>>>>>>>>>>>>>>> symbol looked up: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> core/elf.cc: abort("%s: failed looking up >>>>>>>>>>>>>>>>>>>> symbol %s\n", pathname().c_str(), demangle(name).c_str()); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> You can try to add printouts in >>>>>>>>>>>>>>>>>>>> object::arch_relocate_rela() to try to understand which >>>>>>>>>>>>>>>>>>>> symbol() is being >>>>>>>>>>>>>>>>>>>> called >>>>>>>>>>>>>>>>>>>> with an empty name. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> [backtrace] >>>>>>>>>>>>>>>>>>>>> 0x00000000403442a7 <elf::object::symbol(unsigned int, >>>>>>>>>>>>>>>>>>>>> bool)+391> >>>>>>>>>>>>>>>>>>>>> 0x0000000040397dce >>>>>>>>>>>>>>>>>>>>> <elf::object::arch_relocate_rela(unsigned int, unsigned >>>>>>>>>>>>>>>>>>>>> int, void*, >>>>>>>>>>>>>>>>>>>>> long)+574> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 0x000000004033eed4 <elf::object::relocate_rela()+196> >>>>>>>>>>>>>>>>>>>>> 0x0000000040341d27 <elf::object::relocate()+199> >>>>>>>>>>>>>>>>>>>>> 0x0000000040345623 >>>>>>>>>>>>>>>>>>>>> <elf::program::load_object(std::__cxx11::basic_string<char, >>>>>>>>>>>>>>>>>>>>> std::char_traits<char>, std::allocator<char> >, >>>>>>>>>>>>>>>>>>>>> std::vector<std::__cxx11::basic_string<char, >>>>>>>>>>>>>>>>>>>>> std::char_traits<char>, >>>>>>>>>>>>>>>>>>>>> std::allocator<char> >, >>>>>>>>>>>>>>>>>>>>> std::allocator<std::__cxx11::basic_string<char, >>>>>>>>>>>>>>>>>>>>> std::char_traits<char>, std::allocator<char> > > >, >>>>>>>>>>>>>>>>>>>>> std::vector<std::shared_ptr<elf::object>, >>>>>>>>>>>>>>>>>>>>> std::allocator<std::shared_ptr<elf::object> > >&)+1459> >>>>>>>>>>>>>>>>>>>>> 0x0000000040345e70 >>>>>>>>>>>>>>>>>>>>> <elf::program::get_library(std::__cxx11::basic_string<char, >>>>>>>>>>>>>>>>>>>>> std::char_traits<char>, std::allocator<char> >, >>>>>>>>>>>>>>>>>>>>> std::vector<std::__cxx11::basic_string<char, >>>>>>>>>>>>>>>>>>>>> std::char_traits<char>, >>>>>>>>>>>>>>>>>>>>> std::allocator<char> >, >>>>>>>>>>>>>>>>>>>>> std::allocator<std::__cxx11::basic_string<char, >>>>>>>>>>>>>>>>>>>>> std::char_traits<char>, std::allocator<char> > > >, >>>>>>>>>>>>>>>>>>>>> bool)+336> >>>>>>>>>>>>>>>>>>>>> 0x0000000040465fd8 <dlopen+136> >>>>>>>>>>>>>>>>>>>>> 0x0000100000937228 <_PyImport_FindSharedFuncptr+376> >>>>>>>>>>>>>>>>>>>>> 0x745f70617277796f <???+1920432495> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> It seems the name is not print out. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> If I'm using the check-libcfunc-avail.sh to check the >>>>>>>>>>>>>>>>>>>>> _pywrap_tensorflow_internal.so, I get the following >>>>>>>>>>>>>>>>>>>>> output: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> pthread_mutex_consistent not found >>>>>>>>>>>>>>>>>>>>> pthread_mutexattr_setrobust not found >>>>>>>>>>>>>>>>>>>>> fmaf not found >>>>>>>>>>>>>>>>>>>>> fma not found >>>>>>>>>>>>>>>>>>>>> mallinfo not found >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> All of these we should eventually add, these are real >>>>>>>>>>>>>>>>>>>> Linux glibc functions... >>>>>>>>>>>>>>>>>>>> Feel free to open issues about them, or propose patches >>>>>>>>>>>>>>>>>>>> - some will be trivial (e.g., fma() is already defined in >>>>>>>>>>>>>>>>>>>> musl/ we just >>>>>>>>>>>>>>>>>>>> need to enable it in the Makefile). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 0000000000000000 not found >>>>>>>>>>>>>>>>>>>>> DF not found >>>>>>>>>>>>>>>>>>>>> *UND* not found >>>>>>>>>>>>>>>>>>>>> 0000000000000000 not found >>>>>>>>>>>>>>>>>>>>> (GLIBC_2.2.5) not found >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> These may be (hopefully) parsing errors by the script? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> mremap not found >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Eventually will need to do this too... >>>>>>>>>>>>>>>>>>>> https://github.com/cloudius-systems/osv/issues/184 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Originally, it also contains some output of missing >>>>>>>>>>>>>>>>>>>>> complex number related functions. I include those from >>>>>>>>>>>>>>>>>>>>> musl. The list >>>>>>>>>>>>>>>>>>>>> reduces to the above. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> It would be great to know which symbol it's missing. >>>>>>>>>>>>>>>>>>>>> I'm not sure whether it's looking for a symbol from libc >>>>>>>>>>>>>>>>>>>>> or from other >>>>>>>>>>>>>>>>>>>>> library. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>> Zhiting >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>> You received this message because you are subscribed >>>>>>>>>>>>>>>>>>>>> to the Google Groups "OSv Development" group. >>>>>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving >>>>>>>>>>>>>>>>>>>>> emails from it, send an email to >>>>>>>>>>>>>>>>>>>>> osv...@googlegroups.com. >>>>>>>>>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>>>>>>>>> https://groups.google.com/d/msgid/osv-dev/d84f4991-69d5-45d8-98a2-403257c9e4b2%40googlegroups.com >>>>>>>>>>>>>>>>>>>>> <https://groups.google.com/d/msgid/osv-dev/d84f4991-69d5-45d8-98a2-403257c9e4b2%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>>>>>>>>> . >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>> You received this message because you are subscribed to >>>>>>>>>>>>>>>>>>> the Google Groups "OSv Development" group. >>>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails >>>>>>>>>>>>>>>>>>> from it, send an email to osv...@googlegroups.com. >>>>>>>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>>>>>>> https://groups.google.com/d/msgid/osv-dev/4a72dd0b-b20a-4877-94da-aa9313add35e%40googlegroups.com >>>>>>>>>>>>>>>>>>> <https://groups.google.com/d/msgid/osv-dev/4a72dd0b-b20a-4877-94da-aa9313add35e%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>>>>>>> . >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>>>> Google Groups "OSv Development" group. >>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails >>>>>>>>>>>>>>>> from it, send an email to osv...@googlegroups.com. >>>>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>>>> https://groups.google.com/d/msgid/osv-dev/a83feefd-2c93-48d0-bb9b-674ad0870853%40googlegroups.com >>>>>>>>>>>>>>>> <https://groups.google.com/d/msgid/osv-dev/a83feefd-2c93-48d0-bb9b-674ad0870853%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>>>> . >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "OSv Development" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to osv...@googlegroups.com. >>>>>>>>>>>>> >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/osv-dev/42454aa1-51b9-4638-bcd0-337002364524%40googlegroups.com >>>>>>>>>>>>> <https://groups.google.com/d/msgid/osv-dev/42454aa1-51b9-4638-bcd0-337002364524%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "OSv Development" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to osv...@googlegroups.com. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/osv-dev/648f49e7-6f5a-4340-87ba-c923092c5af6%40googlegroups.com >>>>>>>>>>> <https://groups.google.com/d/msgid/osv-dev/648f49e7-6f5a-4340-87ba-c923092c5af6%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "OSv Development" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to osv...@googlegroups.com. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/osv-dev/CAL9cFfOGj_Jy1VagTiwNXew%3Dd7pyFdbDs7JbvpW88-hqKs27tw%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/osv-dev/CAL9cFfOGj_Jy1VagTiwNXew%3Dd7pyFdbDs7JbvpW88-hqKs27tw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "OSv Development" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to osv-dev+unsubscr...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/osv-dev/e82eb5f7-b3ee-4190-9ee6-7b85df302204%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/osv-dev/e82eb5f7-b3ee-4190-9ee6-7b85df302204%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "OSv Development" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to osv-dev+unsubscr...@googlegroups.com. >>>>>> >>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/osv-dev/F057D3F1-4156-49DE-935E-ECC0218FF401%40gmail.com >>>>>> <https://groups.google.com/d/msgid/osv-dev/F057D3F1-4156-49DE-935E-ECC0218FF401%40gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to osv-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/cc4d7b2b-6cbd-4082-a8cf-22ec7404d38a%40googlegroups.com > <https://groups.google.com/d/msgid/osv-dev/cc4d7b2b-6cbd-4082-a8cf-22ec7404d38a%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CA%2B3q14wcV9Fiph8ie852ZpMYffmJ2dHEvYmMgVaD4sqvh%3D_NUA%40mail.gmail.com.
