Re: Problem with differences with DLOPEN / DLSYM compared to ubuntu (16.04) / debian (stretch).
On 2017-09-15 10:59, Jon Turney wrote: > On 15/09/2017 17:07, cyg Simple wrote: >> Please consider using an interleaving method of posting on this list. >> Top posting is considered rude. >> >> On 9/15/2017 9:51 AM, Gary Schneir wrote: >>> Thanks for the response but I am a little confused by it. If Cygwin is >>> supposed to provide POSIX API functionality and DLOPEN / DLSYM are >>> supported in CYGWIN, then I shouldn't care about the underlying >>> complexity or restrictions of running within the Windows environment and >>> using DLLs. The behavior should be the same as in other POSIX environments. >> >> You presented your case well and I was waiting on someone familiar with >> the code to respond. I'm not sure that would be Kaz, he was just trying >> to be helpful from his experiences. I agree with your surmise that >> Cygwin should perform similar results as Linux in this case. > > ... > >>> If you are saying that I did not include some sort of >>> __declspec(dllexport) directive in my code so that it can find my >>> symbols, that is something else but you indicate that you think cygwin >>> hides that complexity in shared libraries. >> >> Actually it would be binutils, regardless of Cygwin or MinGW, that is >> trying to hide the complexity of needing to supply the >> __declspec([export|import]) declarations. The logic for that is a bit >> confusing but if none is given then all symbols are exported. > > You need to decorate the symbols you wish to be visible with '__attribute__ > ((dllexport))' or '__declspec(dllexport)' (MSVC syntax which is also supported > by gcc) > > See [1] for an example of this done portably > > [1] https://gcc.gnu.org/wiki/Visibility > > Alternatively, you can use the ld flag --export-all-symbols (cf. with the ELF > option --export-dynamic, which I think you must be using to get the observed > behaviour on linux) to make all symbols visible. > > Taking your example, and making it compilable: > > $ cat dlopen.cc > #include > #include > #include > > void * handle, * symbol; > const char * errorStr; > > int main() > { > /* get access to the executable's symbol table */ > handle = dlopen(NULL, RTLD_LAZY); > errorStr = dlerror(); > > if (errorStr) > { > std::clog << "dlopen error '" << errorStr << "'" << std::endl; > } > if (handle) > { > std::clog << "handle ok " << std::endl; > } > else > { > std::clog << "handle NULL " << std::endl; > } > > /* look up the from_string function */ > symbol = dlsym(handle, "functionname"); > errorStr = dlerror(); > > if (symbol) > { > std::clog << "dlsym symbol ok " << std::endl; > } > else > { > std::clog << "dlsym symbol NULL " << std::endl; > } > if (errorStr) > { > std::clog << "dlsym error '" << errorStr << "'" << std::endl; > } > } > > extern "C" __attribute__ ((dllexport)) > void functionname() > { > } > > $ g++ dlopen.cc -o dlopen > > $ ./dlopen > handle ok > dlsym symbol ok No really comparable as the OP example was in a .dll/.so. You would have to make your main e.g test, build into a dll, and call from a separate main program. The issue appears to be that dlopen( NULL, ...) should work as if it was a reference to the dll containing the call, not the main program. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Problem with differences with DLOPEN / DLSYM compared to ubuntu (16.04) / debian (stretch).
On 15.09.2017 06:51, Gary Schneir wrote: Thanks for the response but I am a little confused by it. If Cygwin is supposed to provide POSIX API functionality and DLOPEN / DLSYM are supported in CYGWIN, then I shouldn't care about the underlying complexity or restrictions of running within the Windows environment and using DLLs. The behavior should be the same as in other POSIX environments. I don't see this behavior on Linux. This is a transcript from an Ubuntu 16.04 system: $ txr This is the TXR Lisp interactive listener of TXR 186. Quit with :quit or Ctrl-D on empty line. Ctrl-X ? for cheatsheet. 1> (dlopen nil) # 2> (dlsym *1 "cons") # 3> (dlsym *1 "car") # 4> (dlsym *1 "cdr") # 5> (dlsym *1 "malloc") # 6> (dlsym *1 "printf") # The external functions cons, car and cdr in the txr executable (I assure everyone they are there) cannot be found by dlsym, but malloc and printf in the linked C library are found. I.e. the same issue you're having on Cygwin. Cygwin aims for Linux compatibility more than POSIX. The motto is "get that Linux feeling on Windows", after all. It's splashed right below the Cygwin banner here: https://cygwin.com/ Not finding the executable's own symbol with dlsym (unless some special arrangements are made to dynamically export the symbols) looks very much like a "Linux feeling"! If we run "nm -D" on the executable to see what dynamic symbols are provided, we find only "_init" and "_fini": $ nm -D /usr/local/bin/txr | grep T 080fa3cc T _fini 0804a7a4 T _init and, by golly, these *can* be found with dlsym on the dlopen(NULL, ...) handle! 7> (dlsym *1 "_fini") # 8> (dlsym *1 "_init") # -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Problem with differences with DLOPEN / DLSYM compared to ubuntu (16.04) / debian (stretch).
On 15/09/2017 17:07, cyg Simple wrote: Please consider using an interleaving method of posting on this list. Top posting is considered rude. On 9/15/2017 9:51 AM, Gary Schneir wrote: Thanks for the response but I am a little confused by it. If Cygwin is supposed to provide POSIX API functionality and DLOPEN / DLSYM are supported in CYGWIN, then I shouldn't care about the underlying complexity or restrictions of running within the Windows environment and using DLLs. The behavior should be the same as in other POSIX environments. You presented your case well and I was waiting on someone familiar with the code to respond. I'm not sure that would be Kaz, he was just trying to be helpful from his experiences. I agree with your surmise that Cygwin should perform similar results as Linux in this case. ... If you are saying that I did not include some sort of __declspec(dllexport) directive in my code so that it can find my symbols, that is something else but you indicate that you think cygwin hides that complexity in shared libraries. Actually it would be binutils, regardless of Cygwin or MinGW, that is trying to hide the complexity of needing to supply the __declspec([export|import]) declarations. The logic for that is a bit confusing but if none is given then all symbols are exported. You need to decorate the symbols you wish to be visible with '__attribute__ ((dllexport))' or '__declspec(dllexport)' (MSVC syntax which is also supported by gcc) See [1] for an example of this done portably [1] https://gcc.gnu.org/wiki/Visibility Alternatively, you can use the ld flag --export-all-symbols (cf. with the ELF option --export-dynamic, which I think you must be using to get the observed behaviour on linux) to make all symbols visible. Taking your example, and making it compilable: $ cat dlopen.cc #include #include #include void * handle, * symbol; const char * errorStr; int main() { /* get access to the executable's symbol table */ handle = dlopen(NULL, RTLD_LAZY); errorStr = dlerror(); if (errorStr) { std::clog << "dlopen error '" << errorStr << "'" << std::endl; } if (handle) { std::clog << "handle ok " << std::endl; } else { std::clog << "handle NULL " << std::endl; } /* look up the from_string function */ symbol = dlsym(handle, "functionname"); errorStr = dlerror(); if (symbol) { std::clog << "dlsym symbol ok " << std::endl; } else { std::clog << "dlsym symbol NULL " << std::endl; } if (errorStr) { std::clog << "dlsym error '" << errorStr << "'" << std::endl; } } extern "C" __attribute__ ((dllexport)) void functionname() { } $ g++ dlopen.cc -o dlopen $ ./dlopen handle ok dlsym symbol ok -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Problem with differences with DLOPEN / DLSYM compared to ubuntu (16.04) / debian (stretch).
Please consider using an interleaving method of posting on this list. Top posting is considered rude. On 9/15/2017 9:51 AM, Gary Schneir wrote: > Thanks for the response but I am a little confused by it. If Cygwin is > supposed to provide POSIX API functionality and DLOPEN / DLSYM are > supported in CYGWIN, then I shouldn't care about the underlying > complexity or restrictions of running within the Windows environment and > using DLLs. The behavior should be the same as in other POSIX environments. > You presented your case well and I was waiting on someone familiar with the code to respond. I'm not sure that would be Kaz, he was just trying to be helpful from his experiences. I agree with your surmise that Cygwin should perform similar results as Linux in this case. > > If you are saying that I did not include some sort of > __declspec(dllexport) directive in my code so that it can find my > symbols, that is something else but you indicate that you think cygwin > hides that complexity in shared libraries. > Actually it would be binutils, regardless of Cygwin or MinGW, that is trying to hide the complexity of needing to supply the __declspec([export|import]) declarations. The logic for that is a bit confusing but if none is given then all symbols are exported. -- cyg Simple -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Problem with differences with DLOPEN / DLSYM compared to ubuntu (16.04) / debian (stretch).
Thanks for the response but I am a little confused by it. If Cygwin is supposed to provide POSIX API functionality and DLOPEN / DLSYM are supported in CYGWIN, then I shouldn't care about the underlying complexity or restrictions of running within the Windows environment and using DLLs. The behavior should be the same as in other POSIX environments. If you are saying that I did not include some sort of __declspec(dllexport) directive in my code so that it can find my symbols, that is something else but you indicate that you think cygwin hides that complexity in shared libraries. Gary On 09/14/2017 06:15 PM, Kaz Kylheku wrote: > On 14.09.2017 08:44, Gary Schneir wrote: >> Can anyone provide some assistance to understanding the differences and >> a way get a single code base to work in all three environments without >> naming the library file? > > On Cygwin, libraries are Windows DLL's. That appears to be a > conscious project decision. > > The TL;DR is that Windows DLL's are not going to give you the > behaviors of ELF > shared libraries on GNU/Linux. Cygwin would probably have to whip up > its own > shared library format for that, or port one. > > The complication doesn't seem worth it. > > You can think of dlopen as a wrapper around LoadLibrary, and dlsym > doing GetProcAddress. > > In Windows programming, doing GetProcAddress on symbols within an .exe > is something > that isn't normally done; it's not clear that it is supposed to work > at all. > > For it to even have any hope of working, the function would have to be > somehow > declared "dllexport" (in Visual C terms). > > I think the shared lib building mechanism under Cygwin hides that; > there is no > Visual C __declspec(dllexport) stuff: > > See: https://cygwin.com/cygwin-ug-net/dll.html > > Still, the toolchain has to be arranging for the equivalent of > __declspec(dllexport) > for the external symbols. > > It's probably not configured to do that for the symbols in a regular > executable. > > > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Problem with differences with DLOPEN / DLSYM compared to ubuntu (16.04) / debian (stretch).
On 14.09.2017 08:44, Gary Schneir wrote: Can anyone provide some assistance to understanding the differences and a way get a single code base to work in all three environments without naming the library file? On Cygwin, libraries are Windows DLL's. That appears to be a conscious project decision. The TL;DR is that Windows DLL's are not going to give you the behaviors of ELF shared libraries on GNU/Linux. Cygwin would probably have to whip up its own shared library format for that, or port one. The complication doesn't seem worth it. You can think of dlopen as a wrapper around LoadLibrary, and dlsym doing GetProcAddress. In Windows programming, doing GetProcAddress on symbols within an .exe is something that isn't normally done; it's not clear that it is supposed to work at all. For it to even have any hope of working, the function would have to be somehow declared "dllexport" (in Visual C terms). I think the shared lib building mechanism under Cygwin hides that; there is no Visual C __declspec(dllexport) stuff: See: https://cygwin.com/cygwin-ug-net/dll.html Still, the toolchain has to be arranging for the equivalent of __declspec(dllexport) for the external symbols. It's probably not configured to do that for the symbols in a regular executable. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem with differences with DLOPEN / DLSYM compared to ubuntu (16.04) / debian (stretch).
I am finding a behavior difference with DLOPEN / DLSYM compared to ubuntu (16.04) and debian (stretch), specifically when the DLOPEN is passed NULL for the filename. I have a shared library (.so) file that contains some functions that I need to location by name. The code executing this is within the same .so file. I am trying to avoid naming the file in the DLOPEN statement. According to the documentation, the NULL should indicate the "main program". It is unclear if this is just the executable, the executable and its associated libraries, or the library that is running this function. Ubuntu and Debian, both seem to treat it as able to find within the shared library Regardless of this, there are still differences / issues, if the filename happens to be invalid. ubuntu $ uname -a Linux vm 4.4.0-93-generic #116-Ubuntu SMP Fri Aug 11 21:17:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ g++ --version g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 cygwin $ uname -a CYGWIN_NT-10.0 DESKTOP-FCHFNAG 2.9.0(0.318/5/3) 2017-09-08 10:19 x86_64 Cygwin $ g++ --version g++ (GCC) 6.4.0 debian $ uname -a Linux debian 4.9.0-2-amd64 #1 SMP Debian 4.9.18-1 (2017-03-30) x86_64 GNU/Linux $ g++ --version g++ (Debian 6.3.0-18) 6.3.0 20170516 LD_LIBRARY_PATH is set the same in all three environments. === I have the following code in both ubuntu std::unique_ptr myobject; void * handle, * symbol; const char * errorStr; /* get access to the executable's symbol table */ handle = dlopen(NULL, RTLD_LAZY); errorStr = dlerror(); if (errorStr) { std::clog << " dlopen error '" << errorStr << "'" << std::endl; } if (handle) { std::clog << " handle ok " << std::endl; } else { std::clog << " handle NULL " << std::endl; } /* look up the from_string function */ symbol = dlsym(handle, "functionname"); errorStr = dlerror(); if (symbol) { std::clog << "dlsym symbol ok " << std::endl; } else { std::clog << "dlsym symbol NULL " << std::endl; } if (errorStr) { std::clog << "dlsym error '" << errorStr << "'" << std::endl; } /* * if the associated function is found, the string is parsed. * if not, then a nullptr is returned to the caller. */ if (symbol) { std::clog << " calling symbol " << std::endl; myobject = (*(from_string_type *)symbol)(str); } . If I pass a NULL parameter to DLOPEN - different behavior handle = dlopen(NULL, RTLD_LAZY) under ubuntu handle ok dlsym symbol ok calling symbol >> Correctly finds symbol and works correctly. under cygwin handle ok dlsym symbol NULL dlsym error 'No such process' >> Cannot find symbol and therefore fails under debian handle ok dlsym symbol ok calling symbol >> Correctly finds symbol and works correctly. If I pass an invalid library name parameter to DLOPEN - different behvior handle = dlopen("invalidname", RTLD_LAZY) On ubuntu, dlopen error 'invalidname: cannot open shared object file: No such file or directory' handle NULL dlsym symbol ok calling symbol Segmentation fault (core dumped) >> Symbol was not correct, even though it was not null an no error was returned on cygwin, dlopen error 'No such file or directory' handle NULL dlsym symbol ok calling symbol >> Correctly finds symbol and works correctly. On debian, dlopen error 'invalidname: cannot open shared object file: No such file or directory' handle NULL dlsym symbol ok calling symbol Segmentation fault >> Symbol was not correct, even though it was not null an no error was returned If I pass an correct library name parameter to DLOPEN - same behvior handle = dlopen("libmine.so", RTLD_LAZY) On ubuntu, handle ok dlsym symbol ok calling symbol >> Correctly finds symbol and works correctly. on cygwin, handle ok dlsym symbol ok calling symbol >> Correctly finds symbol and works correctly. On debian, handle ok dlsym symbol ok calling symbol >> Correctly finds symbol and works correctly. Can anyone provide some assistance to understanding the differences and a way get a single code base to work in all three environments without naming the library file? Thanks -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple