Re: autotools not suited to proprietary development?
On Wed, 2006-10-04 at 21:33 -0700, Andre Stechert wrote: On Oct 4, 2006, at 8:55 PM, Ryan McDougall wrote: However the problem remains that Im at a bit of a loss how to ship a shared .SO library easily. If I build on my machine (or a set of supported build machines) then the build will link to my prefix (lets say /usr/lib) and the end user has no choice where he wants to place his library? This can be hard to do portably but not because of the autotools. I think it boils down to two kinds of problems: 1) If your build is libtoolized, then you may end up with -rpath references in your .la files (Google for debian rpath to get a sense of the drama that's followed this issue over the years). If you're just distributing the .so's, then this isn't so much of a problem, as you can fix it by having your users install the .so wherever they want, but they have to reference by augmenting their LD_LIBRARY_PATH appropriately. In summary, if you're careful about it, you can do exactly what you propose. Cheers, Andre Apologies if this is starting to wander off course, but Im really trying to learn all this stuff so I can port more programs to linux, so I hope you dont mind being the ones to give me a clue but ... should I understand that (for example) when redhat/debian build lets say libc for packaging as a binary, they download a tarball and do a complicated form './configure --prefix=/usr make make install' on a bare machine without any libc, then tar up the result for an RPM or DEB? I have looked briefly how to make RPMs, but Ive got some missing pieces here... Im well aware this is getting offtopic, so if possible could you spare a clue and a link to a place where I can research the problem more myself? Cheers,
autotools not suited to proprietary development?
Hello, First of all, Im only looking for serious answers, and I dont want to start any flamewars, so Ill start out by saying that quite likely the only problem here is my lack of knowledge of autotools, and the other the issues involved. Please correct me at any point you disagree with. After some cajoling of new company from myself -- a linux and free software buff, and a lonely professor who wants to use one of our SDKs on linux, it was finally decided to make a linux port of our SmartCollision SDK. The C++ code is clean and portable, so porting really just consists of building the binaries, and putting them in a friendly distributable. In windows the deliverable is a zip file with .DLLs, .LIBs, and header files that should work on any modern widows. The linux deliverable should be the same, however, given the large amount of variability in the linux world, I would like to have autoconf do as much work as possible, which I think means getting the code to compile on my build machine, and link on the users environment. With my inexperience with doing proprietary development, I chose to use autotools since I was the only tool I was familiar with from open source projects. The problem is the major assumption in autotools is that the source code is being shipped to the end user, and thus the distributable package is the same as the source tarball. I was unable, in the time allowed, to come up with a way to easily (ie clean, without many hacks) give me the sort of solution I was looking for, which would look something like: './configure make make bindist' on the build machine and './configure make make install' on the end users machine As it is we just './configure --prefix=/package/dir/ make make install' then manually package the static libA.a libraries and headers there. We couldnt do shared libraries since libtool wanted to link the .so to the build machine's prefix (via -rpath switch). Has this problem been solved before with autotools? Or is this just not a problem autotools wants to solve? If so, does anyone have alternative recommendations? I ask because I have hope we can port our remaining products to linux, and that I can learn how to do this sort of thing properly next time. But if it gets too expensive for my boss, he wont be likely to approve it. Cheers,
Re: autotools not suited to proprietary development?
On Wed, 2006-10-04 at 20:11 -0700, Andre Stechert wrote: Hi, I think you're confusing the idea of a build system for portable software (something the autotools suite can help with) and an installer (or package if you're installing onto a system that has a package manager). From a Windows perspective, it's the same as the difference between Visual Studio and something like InstallShield or NSIS. I think is is quite true, years of free software having skewed my experiences and spoiled me to easy source-based portability. :) However the problem remains that Im at a bit of a loss how to ship a shared .SO library easily. If I build on my machine (or a set of supported build machines) then the build will link to my prefix (lets say /usr/lib) and the end user has no choice where he wants to place his library? This is the kind of detail I would very much hope was solved by some software somewhere... Cheers, One trick that may help is automake's DESTDIR variable, which allows you to create a degenerate package (i.e., something that doesn't allow you to do fancy system setup -- not unlike installing a Windows app from a .zip file). When you're doing make install, try instead: make install DESTDIR=/tmp/staging Automake will build everything as if it's going to install to your -- prefix (including embedding references to your prefix in the appropriate places) and then at the last minute, it will take the files it was going to install and destdir them at /tmp/staging. Then you can simply tar/gz the files at /tmp/ staging and ship those to your customers. Or zip them. Cheers, Andre On Oct 4, 2006, at 7:41 PM, Ryan McDougall wrote: Hello, First of all, Im only looking for serious answers, and I dont want to start any flamewars, so Ill start out by saying that quite likely the only problem here is my lack of knowledge of autotools, and the other the issues involved. Please correct me at any point you disagree with. After some cajoling of new company from myself -- a linux and free software buff, and a lonely professor who wants to use one of our SDKs on linux, it was finally decided to make a linux port of our SmartCollision SDK. The C++ code is clean and portable, so porting really just consists of building the binaries, and putting them in a friendly distributable. In windows the deliverable is a zip file with .DLLs, .LIBs, and header files that should work on any modern widows. The linux deliverable should be the same, however, given the large amount of variability in the linux world, I would like to have autoconf do as much work as possible, which I think means getting the code to compile on my build machine, and link on the users environment. With my inexperience with doing proprietary development, I chose to use autotools since I was the only tool I was familiar with from open source projects. The problem is the major assumption in autotools is that the source code is being shipped to the end user, and thus the distributable package is the same as the source tarball. I was unable, in the time allowed, to come up with a way to easily (ie clean, without many hacks) give me the sort of solution I was looking for, which would look something like: './configure make make bindist' on the build machine and './configure make make install' on the end users machine As it is we just './configure --prefix=/package/dir/ make make install' then manually package the static libA.a libraries and headers there. We couldnt do shared libraries since libtool wanted to link the .so to the build machine's prefix (via -rpath switch). Has this problem been solved before with autotools? Or is this just not a problem autotools wants to solve? If so, does anyone have alternative recommendations? I ask because I have hope we can port our remaining products to linux, and that I can learn how to do this sort of thing properly next time. But if it gets too expensive for my boss, he wont be likely to approve it. Cheers,
support for binary dist?
Hello, I am porting a proprietary (yes I know, but it pays the bills) windows application to linux using autotools. So far I have gotten automake to do what I want up until the install/build stage. I wish to provide our end-users with a tarball that includes a dynamically linkable library, and the needed header files for our SDK. As far as I can tell, I have two options: 1. make install into a package directory, then tar that up by hand 2. make dist while somehow excluding the original .CPP files. Which immediately gives me two problems: 1. make install links my dynamic SO to my package directory instead of the user's prefix directory 2. make dist complains about the missing .CPP files, even if I include all the .O and .LO files Does automake have any make bindist support? Can anyone recommend a method to me? I dont have much experience with making proprietary (non-GNU) applications so I apologize if I missed something. Cheers,
Re: The automake/libtool way to make a dynamic lib from multiple sources
On Thu, 2006-09-28 at 13:06 +0200, Ralf Wildenhues wrote: Hello Ryan, * Ryan McDougall wrote on Thu, Sep 28, 2006 at 11:30:32AM CEST: On Thu, 2006-09-28 at 18:23 +0900, Ryan McDougall wrote: How can I either 1. Express to automake the correct dependancy By writing libA_la_LIBADD = libfoo.la instead of libA_la_LIBADD = -L. -lfoo I just discovered that I can trick automake with this: zardir=/dev/null zar_LTLIBRARIES=libfoo.la lib_LTLIBRARIES=libA.la and this will allow libfoo to be built before libA, however this is sort of trickery is bound to bite me in the ass somewhere, right? That is wrong, as it causes libfoo to not be a convenience archive, but a real, installable library. It's doomed to failure, as the installed libA may depend on it being present. Cheers, Ralf Cheers, everything works as I need it to to get the basic build up! Thanks for all your help Ralf! For the sake of those that google after me, here is my solution: In the directory containing the target library directories, and the dependent library directories I have the following Makefile.am AUTOMAKE_OPTIONS = subdir-objects noinst_LTLIBRARIES = lib_LTLIBRARIES = include libfoo/Makefile.am include libbar/Makefile.am include libA/Makefile.am within the dependent library directories I have the following Makefile.am noinst_LTLIBRARIES += libfoo.la libfoo_la_CPPFLAGS = -Ilibfoo/include libfoo_la_SOURCES = libfoo/src/libfoo.cpp within the target library directories I have the following Makefile.am lib_LTLIBRARIES += libA.la libA_la_LIBADD = libfoo.la libA_la_CPPFLAGS = -DIMPORTANT_COMPILE_TIME_SWITCHES \ -IlibA/include \ -Ilibfoo/include libA_la_SOURCES = libA/src/libA.cpp
Re: The automake/libtool way to make a dynamic lib from multiple sources
On Fri, 2006-09-22 at 04:34 +0200, Ralf Wildenhues wrote: * Ryan McDougall wrote on Fri, Sep 22, 2006 at 02:54:21AM CEST: On Thu, 2006-09-21 at 13:32 +0200, Ralf Wildenhues wrote: * Ryan McDougall wrote on Thu, Sep 21, 2006 at 11:40:37AM CEST: 2) If you prefer nonrecursive makefiles (you may include makefile snippets from one into another), then you can just AUTOMAKE_OPTIONS = subdir-objects module_LTLIBRARIES = libA.la libB.la libC.la libA_la_SOURCES = foo/foo1.c bar/bar2.c A/a1.c ... libB_la_SOURCES = baz/baz1/c bar/bar2.c B/b1.c .. I thought of this, but in this case libA_la_SOURCES could be on the order of 100-50 files long, which Id like to avoid if possible. You don't have to set this in one, and you can use variables (and Automake conditionals) to factor. For example: -- Makefile.am -- AUTOMAKE_OPTIONS = subdir-objects module_LTLIBRARIES = libA.la libB.la libC.la libA_la_SOURCES = libB_la_SOURCES = common_sources = include foo/snippet.am include bar/snippet.am if CONDITION_BAZ libB_la_SOURCES += baz/baz1.c endif libA_la_SOURCES += $(common_sources) libB_la_SOURCES += $(common_sources) -- foo/snippet.am -- ## Note the file name including the subdir here: we are being included ## from a level up (Automake includes just do literal replacement): libA_la_SOURCES += foo/foo1.c -- bar/snippet.am common_sources += \ bar/bar2.c \ ... Thanks for all your help Ralf, I _really_ appreciate you taking your time to help me out. I am currently attempting to use a version of this (non-recursive) method. I am avoiding exactly what you wrote since I need to pass specific -D options to different packages without using libA_CPPFLAGS = -DFORLIBFOO -DFORLIBBAR -DETC. What I have now is I have replaced the file names in the recursive Makefile.am's to be relative to the root directory like so: libfoo_SOURCES=src/foo.cpp - libfoo_SOURCES=libfoo/src/foo.cpp and included libfoo/Makefile.am in the root Makefile.am. This works well, and would nearly be finished, except in my root Makefile.am I get this: noinst_LTLIBRARIES = libfoo.la ... lib_LTLIBRARIES = libA.la which Automake dutifully turns into a rule which always builds libA.la before libfoo.la, despite the fact that libA depends on libfoo (as expressed by the variable libA_la_LIBADD = -lfoo). How can I either 1. Express to automake the correct dependancy 2. Trick automake into building lib_LTLIBRARIES last (after noinst_LTLIBRARIES) ps. This would have been a lot easier if Automake would have automatically provided targets for each one of the SUBDIRS. That way I could have simply wrote one rule, libA: foo bar baz (where foo bar baz are SUBDIRs that need to be build before the SUBDIR libA). Should I open a bug/feature request? Thanks a tonne for your unheralded work!!! Id hate to have to give up on autotools. Cheers,
Re: The automake/libtool way to make a dynamic lib from multiple sources
On Thu, 2006-09-28 at 18:23 +0900, Ryan McDougall wrote: On Fri, 2006-09-22 at 04:34 +0200, Ralf Wildenhues wrote: * Ryan McDougall wrote on Fri, Sep 22, 2006 at 02:54:21AM CEST: On Thu, 2006-09-21 at 13:32 +0200, Ralf Wildenhues wrote: * Ryan McDougall wrote on Thu, Sep 21, 2006 at 11:40:37AM CEST: How can I either 1. Express to automake the correct dependancy 2. Trick automake into building lib_LTLIBRARIES last (after noinst_LTLIBRARIES) I just discovered that I can trick automake with this: zardir=/dev/null zar_LTLIBRARIES=libfoo.la lib_LTLIBRARIES=libA.la and this will allow libfoo to be built before libA, however this is sort of trickery is bound to bite me in the ass somewhere, right? Cheers,
The automake/libtool way to make a dynamic lib from multiple sources
Hello, I apologize if this has been answered somewhere. Ive read the book and searched the archives. My situation is this: Im porting a large code-base from windows to linux, and attempting to use autotools to do so. The code in SVN is essentially one big tree of somewhat related modules -- lets call them foo, bar, etc (Projects under MS VS). I need to produce 3 dynamic libraries that can be built from those modules -- lets call them A.so, B.so, and C.so (Solutions under VS). They depend on a non-overlapping, non-disjoint set of the source code modules in the directory. Directories for A, B, and C also contain .cpp files and appear in the same parent directory as foo, bar, etc. Currently I have all directories building respective foo.las -- including A.la, B.la, and C.la, and have them installing correctly. Problems: - I dont need the foo,bar,etc constituent libraries, I just need A,B,C libraries, which means I have to manually delete them from the install location afterwords - I would like to be able to type Make A from the parent directory, and have the build system figure out which among foo,bar,etc libraries are needed by A and build them in the manner you would expect - I would like to accomplish this with a minimum of messing with the file hierarchy. Since Visual Studio can manage this feat, Im hoping its not above autotools! What do you recommend great wizards of autotoolry? Cheers,
Re: The automake/libtool way to make a dynamic lib from multiple sources
On Thu, 2006-09-21 at 13:32 +0200, Ralf Wildenhues wrote: Hello Ryan, * Ryan McDougall wrote on Thu, Sep 21, 2006 at 11:40:37AM CEST: Im porting a large code-base from windows to linux, and attempting to use autotools to do so. The code in SVN is essentially one big tree of somewhat related modules -- lets call them foo, bar, etc (Projects under MS VS). I need to produce 3 dynamic libraries that can be built from those modules -- lets call them A.so, B.so, and C.so (Solutions under VS). They depend on a non-overlapping, non-disjoint set of the source code modules in the directory. non-overlapping, non-disjoint sounds a bit like a typo; is it? Or do you mean they just partially overlap? Yes, that was me being tired and not having my brain entirely in English. :) Directories for A, B, and C also contain .cpp files and appear in the same parent directory as foo, bar, etc. Currently I have all directories building respective foo.las -- including A.la, B.la, and C.la, and have them installing correctly. Problems: - I dont need the foo,bar,etc constituent libraries, I just need A,B,C libraries, which means I have to manually delete them from the install location afterwords 2 possibilities: 1) If you prefer recursive makefiles then you can have each directory create one (or a number of) libtool convenience archives (noinst_LTLIBRARIES), which you can merge into the real modules (A_la_LIBADD = ../foo/libfoo.la) You should keep in mind that the convenience archives will end up in total (i.e., all their symbols, not just needed ones) in the resulting module. This is a good point, thank you. Do you have any more information somewhere that elaborates on this point? 2) If you prefer nonrecursive makefiles (you may include makefile snippets from one into another), then you can just AUTOMAKE_OPTIONS = subdir-objects module_LTLIBRARIES = libA.la libB.la libC.la libA_la_SOURCES = foo/foo1.c bar/bar2.c A/a1.c ... libB_la_SOURCES = baz/baz1/c bar/bar2.c B/b1.c .. I thought of this, but in this case libA_la_SOURCES could be on the order of 100-50 files long, which Id like to avoid if possible. AUTOMAKE_OPTIONS=subdir-objects tells automake to build temporary objects (foo.lo) in the sub directories to improve build times? What is the module target? - I would like to be able to type Make A from the parent directory, and have the build system figure out which among foo,bar,etc libraries are needed by A and build them in the manner you would expect The second approach will enable you to make libA.la in the toplevel directory. Otherwise, there is nothing that prevents you from adding your own targets a la A: cd src/A $(MAKE) $(AM_MAKEFLAGS) libA.la .PHONY: A just as in ordinary makefiles. I already have this done, but I was hoping the was a more magic way to do this. Cheers, Ralf Thanks for all your help Ralf! Cheers,
Re: The automake/libtool way to make a dynamic lib from multiple sources
On Thu, 2006-09-21 at 13:32 +0200, Ralf Wildenhues wrote: The second approach will enable you to make libA.la in the toplevel directory. Otherwise, there is nothing that prevents you from adding your own targets a la A: cd src/A $(MAKE) $(AM_MAKEFLAGS) libA.la .PHONY: A just as in ordinary makefiles. Cheers, Ralf The problem with this is that with this method that I have to manually specify the dependencies, right? A: foo cd src/A $(MAKE) $(AM_MAKEFLAGS) libA.la .PHONY: A foo: cd src/foo $(MAKE) $(AM_MAKEFLAGS) libfoo.la .PHONY: foo etc ... Cheers,
Re: The automake/libtool way to make a dynamic lib from multiple sources
On Fri, 2006-09-22 at 04:34 +0200, Ralf Wildenhues wrote: You should keep in mind that the convenience archives will end up in total (i.e., all their symbols, not just needed ones) in the resulting module. This is a good point, thank you. Do you have any more information somewhere that elaborates on this point? The Libtool manual has more details about this (but it's arguably a bit cryptic about convenience archives). I'm not sure what you need to know. Since Im leaning to this method, Id like to investigate just what sort of inefficiencies Im going to incur this way. Basically I don't know a whole lot about the details of dynamic linking, so Id like to learn whats happening, if possible/reasonable. What is the module target? Oh, I forgot. If you install the modules in $(libdir), just use lib_LTLIBRARIES; for the above to work you need something like moduledir = $(libdir)/foo or similar (e.g., pkglibdir; or a setting in configure.ac). You should not forget the libtool link flag -module (libA_la_LDFLAGS). This sounds interesting, is there a place I can read about this modules bit? Cheers, Ralf Cheers,
