Re: linking problems with several static libraries
On שבת, 13 ביולי 2013 21:42:12 Valery Reznic wrote: > >And I was under impression that "ld" was supposed to handle such kind of > >cyclic dependencies. > > ld indeed able to handle it. > > From ld man page: > -( archives -) --start-group archives --end-group The archives should be a > list of archive files. They may be either explicit file names, or -l > options. The specified archives are searched repeatedly until no new > unde- fined references are created. Normally, an archive is searched > only once in the order that it is specified on the command line. If a > symbol in that archive is needed to resolve an undefined sym- bol referred > to by an object in an archive that appears later on the command line, > the linker would not be able to resolve that ref- erence. By grouping the > archives, they all be searched repeatedly until all possible references > are resolved. Using this option has a significant performance cost. It > is best to use it only when there are unavoidable circular references > between two or more archives. Regards, > Valery mmm.. here lies another problem. I am using ndk-build, which kinda prevents you from using direcly the LD command. You tell it which libraries to link and it creates the LD/CXX command. I ended hackign inside LOCAL_LDFLAGS, which worked. I also tried using -Wl,--whole-archive and this is what I got: /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.3/libgcc.a(_fixsfdi.o) (.text+0x2e): unresolvable R_X86_64_PLT32 relocation against symbol `__fixunssfti@@GCC_3.0' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.3/libgcc.a(_fixsfdi.o) (.text+0x12): unresolvable R_X86_64_PLT32 relocation against symbol `__fixunssfti@@GCC_3.0' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.3/libgcc.a(_fixdfdi.o) (.text+0x2f): unresolvable R_X86_64_PLT32 relocation against symbol `__fixunsdfti@@GCC_3.0' and also some references to symbols inside one of the libraries. Again - wierd. Anyway, I can only say that ndk-build is a toy build-system, as much as qmake. It works, until it breaks and you need to workaround it. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: linking problems with several static libraries
On יום רביעי, 10 ביולי 2013 21:43:54 Omer Zak wrote: > On Wed, 2013-07-10 at 21:31 +0300, Diego Iastrubni wrote: > > My program needs liba, which in turn needs libb.. which in turn needs > > libk. The last libk needs symbols from liba.. and this is where it gets > > funky. While linking g++ complains that symbols are missing ... and from > > ar+nm I see that those symbols are avarilable on liba. > > You did not say if those libraries are your employer's or provided by > 3rd parties. > > If the libraries are your employer's, then the long range solution is to > restructure (another term is refactor) them to eliminate the circular > dependencies from which you suffer. > > This would also make it possible to develop unit tests for the libraries > and for your project in general. > > If the libraries are provided by a 3rd party, then report a bug (!) > against them, and have them refactor the libraries to fix the bug. All libraries are 3rd party libs. Even the parts of code I have are obsfucated. Really nice trick. And I was under impression that "ld" was supposed to handle such kind of cyclic dependencies. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: linking problems with several static libraries
Oron Peled writes: > On Wednesday 10 July 2013 21:31:51 Diego Iastrubni wrote: >> >> My program needs liba, which in turn needs libb.. which in turn needs libk. >> The last libk needs symbols from liba.. and this is where it gets funky. >> While linking g++ complains that symbols are missing ... and from ar+nm I >> see that those symbols are avarilable on liba. > > * Omer Zak correctly replied that this circular dependency represent >bad design of the libraries authors. > > * But don't despair yet... I think there is yet another option that may help in certain cases - I have not used it for a while and I am writing from memory, which may be faulty. Check the docs, try, test. There may be a symbol (or a few) in your libk - or even earlier in the cycle - that will be unresolved without another pass at liba, but your program may not need this symbol at all. In such a case you can add "-u " or "--undefined=" to the ld command (or pass from gcc) to keep the symbol unresolved in the executable. If it is never used - who cares? You may repeat the option for multiple symbols. At the risk of repetition - but I think it can't be emphasized enough: if these (at least liba and libk) are your in-house libraries do what you can to force your co-workers/managers/etc. to figure out what should properly go into each of them and redesign. What you see is a symptom of not enough thought invested in modelling the domain problem or proper modularization of functionality or both. This will likely bite your team in the future in more serious ways than this. So deal with it early. -- Oleg Goldshmidt | p...@goldshmidt.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: linking problems with several static libraries
On Thu, Jul 11, 2013 at 1:39 AM, Oron Peled wrote: > > On Wednesday 10 July 2013 21:31:51 Diego Iastrubni wrote: > > I have been figthing this nice problem at work, which I would like > someone > > to help me. Basically, I have several static libs (liba... libk) which I > > need to link to my program. > > > > My program needs liba, which in turn needs libb.. which in turn needs > libk. > > The last libk needs symbols from liba.. and this is where it gets funky. > > While linking g++ complains that symbols are missing ... and from ar+nm I > > see that those symbols are avarilable on liba. > > * Omer Zak correctly replied that this circular dependency represent >bad design of the libraries authors. > > * But don't despair yet... see below. > > > My solution was to > > > > g++ -o blabla $(OBJS) liba.a libb.a l...libk.a liba.a > > > > I know that using -L -l does not work as well, at I do need to link > liba.a > > twice. > > * It has nothing to do with '-L' which just adds directories to the search > path > > * It's because the linker in Linux (and all Unix systems I've encountered) > is >a single pass one. > > * So in general the linking *order* matters -- and circular dependency > suck. > > * But... there are several workarounds for such brain-dead situations. > > * Workaround 1: > - List libraries multiple times, like you did > - It does exactly what you want > - But if the dependency graph is more complex... you'd have to > work harder to find the correct repetitions and order :-( > > * Workaround 2: > - Pass the '--whole-archive' option to 'ld' > If you link via gcc, just tell it to pass this option to the linker > via '-Wl' option: > '-Wl,--whole-archive' > > - You don't need to think at all about all these cycles, but > > - The complete set of libraries gets into your executable (even > unused data and code) -- so you typically generate very big > executable (yes, doing things the dumb way has its price). > > Continuing this line of thought, you could create (once) one big static archive from all of the smaller ones with ar. Then you will only be linking each time with one thing. You can use ar to do that on the basis of the static libraries, no need to compile everything together. Then you will link with it and create an executable which is exactly the right size. > * Workaround 3: >- This is a "partial" workaround 2 solution. > >- Let's say only a subset of the libraries has this cyclic > dependency problem > >- Than you can use the linker '--start-group', '--end-group' options: > ld libx.a liby.a --start-group libbad1.a libbad2.a --end-group > libz.a > > Or, let the programmers deal with the results of their bad design and > reorganize the code in the libraries -- after all a library should contain > related and well defined functionality -- the cycles just shows that > somebody just threw pieces of code to different libraries. > > Enjoy, > > -- > Oron Peled Voice: +972-4-8228492 > o...@actcom.co.il http://users.actcom.co.il/~oron > UNIX is user friendly. It's just selective about who its friends are. > > > ___ > Linux-il mailing list > Linux-il@cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > -- Orna Agmon Ben-Yehuda. http://ladypine.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: linking problems with several static libraries
On Wednesday 10 July 2013 21:31:51 Diego Iastrubni wrote: > I have been figthing this nice problem at work, which I would like someone > to help me. Basically, I have several static libs (liba... libk) which I > need to link to my program. > > My program needs liba, which in turn needs libb.. which in turn needs libk. > The last libk needs symbols from liba.. and this is where it gets funky. > While linking g++ complains that symbols are missing ... and from ar+nm I > see that those symbols are avarilable on liba. * Omer Zak correctly replied that this circular dependency represent bad design of the libraries authors. * But don't despair yet... see below. > My solution was to > > g++ -o blabla $(OBJS) liba.a libb.a l...libk.a liba.a > > I know that using -L -l does not work as well, at I do need to link liba.a > twice. * It has nothing to do with '-L' which just adds directories to the search path * It's because the linker in Linux (and all Unix systems I've encountered) is a single pass one. * So in general the linking *order* matters -- and circular dependency suck. * But... there are several workarounds for such brain-dead situations. * Workaround 1: - List libraries multiple times, like you did - It does exactly what you want - But if the dependency graph is more complex... you'd have to work harder to find the correct repetitions and order :-( * Workaround 2: - Pass the '--whole-archive' option to 'ld' If you link via gcc, just tell it to pass this option to the linker via '-Wl' option: '-Wl,--whole-archive' - You don't need to think at all about all these cycles, but - The complete set of libraries gets into your executable (even unused data and code) -- so you typically generate very big executable (yes, doing things the dumb way has its price). * Workaround 3: - This is a "partial" workaround 2 solution. - Let's say only a subset of the libraries has this cyclic dependency problem - Than you can use the linker '--start-group', '--end-group' options: ld libx.a liby.a --start-group libbad1.a libbad2.a --end-group libz.a Or, let the programmers deal with the results of their bad design and reorganize the code in the libraries -- after all a library should contain related and well defined functionality -- the cycles just shows that somebody just threw pieces of code to different libraries. Enjoy, -- Oron Peled Voice: +972-4-8228492 o...@actcom.co.il http://users.actcom.co.il/~oron UNIX is user friendly. It's just selective about who its friends are. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: linking problems with several static libraries
read this tutorial fully - and you should get your answer: http://users.actcom.co.il/~choo/lupg/tutorials/libraries/unix-c-libraries.html in particular, the answer is in section 8.1 --guy On 07/10/2013 09:31 PM, Diego Iastrubni wrote: Hi all, I have been figthing this nice problem at work, which I would like someone to help me. Basically, I have several static libs (liba... libk) which I need to link to my program. My program needs liba, which in turn needs libb.. which in turn needs libk. The last libk needs symbols from liba.. and this is where it gets funky. While linking g++ complains that symbols are missing ... and from ar+nm I see that those symbols are avarilable on liba. My solution was to g++ -o blabla $(OBJS) liba.a libb.a l...libk.a liba.a I know that using -L -l does not work as well, at I do need to link liba.a twice. I can reproduce this on Ubuntu 10.04 64bit (default toolchain, gcc 4.6) and in Android's NDK (using all availble toolchains 4.4.3, 4.6 and clang 3.2). Can anyone explain why am I seeing this problem? - diego ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: linking problems with several static libraries
On Wed, 2013-07-10 at 21:31 +0300, Diego Iastrubni wrote: > My program needs liba, which in turn needs libb.. which in turn needs libk. > The last libk needs symbols from liba.. and this is where it gets funky. > While > linking g++ complains that symbols are missing ... and from ar+nm I see that > those symbols are avarilable on liba. You did not say if those libraries are your employer's or provided by 3rd parties. If the libraries are your employer's, then the long range solution is to restructure (another term is refactor) them to eliminate the circular dependencies from which you suffer. This would also make it possible to develop unit tests for the libraries and for your project in general. If the libraries are provided by a 3rd party, then report a bug (!) against them, and have them refactor the libraries to fix the bug. --- Omer -- There is no IGLU Cabal because. My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
