Re: [apparmor] [PATCH] towards a common build infrastructure
Hi Kees, Thanks for starting this conversation; I'd been thinking about this as well for 2.8+1. On Sun, May 06, 2012 at 06:50:27PM -0700, Kees Cook wrote: > On Mon, May 07, 2012 at 12:09:08AM +, Seth Arnold wrote: > > Use $MAKE instead, that way the jobserver can keep informed about how many > > jobs sub-makes are using. > > > > I do like that ./configure --prefix makes selecting between /usr and > > /usr/local easy but I have trouble seeing what else it gives us. > > Sanity for shared library creation (libtool, pkg-config, etc), automatic > handling of finding functions (e.g. I'm building on Debian kfreebsd...), > and finding dep version (e.g. swig, perl, python, ruby, apache). Trying to > do all this by hand is a waste of time -- all of the needed logic already > exists in autoconf/automake. While that's mostly true, while trying to enable the libapparmor python library, I found the intersection of autotools + swig + dh_python2 inscrutable in that I could not figure out what magic was required to build for more than one python version. I *think* part of the issue there was that while it's easy to test for the existence of swig binaries, autotools macro support for swig is not well integrated into the default set of macros. Historical issues have been that I've tended to use toplevel make targets to deal with release management in consistent way. Needing to run the autotools stuff before that can be done feels suboptimal. These could potentially be shove to shell scripts, but again, less than ideal. But that said, I think we should move toward a standardized tooling infrastructure. Whether that's autotools or cmake, I'd like to hear opinions. > > And now tell me how automake can make things simpler than this ;-) > > I know it'll be hard to convince you, but without being too flippant, the > automake version of your example is a single line, just "SUBDIRS = ...". :) For people that are not familiar with autotools, I highly recommend the tutorial at: http://www.lrde.epita.fr/~adl/autotools.html It explained a lot of autotools concepts and usage in ways that I had not seen clearly explained before. -- Steve Beattie http://NxNW.org/~steve/ signature.asc Description: Digital signature -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
Re: [apparmor] [PATCH] towards a common build infrastructure
On Mon, May 07, 2012 at 12:09:08AM +, Seth Arnold wrote: > Use $MAKE instead, that way the jobserver can keep informed about how many > jobs sub-makes are using. > > I do like that ./configure --prefix makes selecting between /usr and > /usr/local easy but I have trouble seeing what else it gives us. Sanity for shared library creation (libtool, pkg-config, etc), automatic handling of finding functions (e.g. I'm building on Debian kfreebsd...), and finding dep version (e.g. swig, perl, python, ruby, apache). Trying to do all this by hand is a waste of time -- all of the needed logic already exists in autoconf/automake. > From: Christian Boltz > [...] > > I'd guess even more people understand plain Makefiles ;-) and I can > guarantee you that nearly nobody understands the automake-generated > Makefiles (reading them might be needed to track down build issues) It's extremely rare to need to look at anything by the Makefile.am. > Well, I remember (from another project) that plain Makefiles can work > with subdirs without problems. Something like this will work as Makefile > in the top-level directory (untested braindump) Right. When I redesigned the Inkscape build system, the final result was a single top-level Makefile (generated by autoconf and automake), that allowed for full parallelization of the build, etc. It was extremely nice. We can easily get there, but doing the migration is going to take a few steps. > And now tell me how automake can make things simpler than this ;-) I know it'll be hard to convince you, but without being too flippant, the automake version of your example is a single line, just "SUBDIRS = ...". :) -Kees -- Kees Cook -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
Re: [apparmor] [PATCH] towards a common build infrastructure
Use $MAKE instead, that way the jobserver can keep informed about how many jobs sub-makes are using. I do like that ./configure --prefix makes selecting between /usr and /usr/local easy but I have trouble seeing what else it gives us. -Original Message- From: Christian Boltz Sender: apparmor-boun...@lists.ubuntu.com Date: Mon, 07 May 2012 01:12:20 To: Subject: Re: [apparmor] [PATCH] towards a common build infrastructure Hello, Am Sonntag, 6. Mai 2012 schrieb Kees Cook: > On Sun, May 06, 2012 at 01:46:30AM +0200, Christian Boltz wrote: > > I prefer hand-written Makefiles - but that might be a matter of > > personal taste ;-) (and, in my case, missing knownledge about > > automake) > > I find it much easier to deal with large projects via autoconf and > automake. There is rarely anything 'strange' enough that it requires > much fiddling. I prefer it because it makes the build system familiar > to most people trying to understand it, and the templates are small. I'd guess even more people understand plain Makefiles ;-) and I can guarantee you that nearly nobody understands the automake-generated Makefiles (reading them might be needed to track down build issues) > > > === modified file 'README' > > > > You only deleted the lines describing the current build "system", > > but > > didn't add any note how to build AppArmor with automake... > > It's still there. It's just hugely reduced. (Which is why I wanted to > push for this.) Re-reading your patch a second time, I noticed that you are right. I expected some + lines in the README patch, but you found a clever way to recycle the existing lines ;-) > I just want the build system simplified. Well, I remember (from another project) that plain Makefiles can work with subdirs without problems. Something like this will work as Makefile in the top-level directory (untested braindump) subdirs=changehat/mod_apparmor changehat/pam_apparmor libraries/libapparmor parser profiles utils all: for dir in $(subdirs) ; do make -C $$dir || exit 1 ; done clean: for dir in $(subdirs) ; do make -C $$dir clean || exit 1 ; done install: for dir in $(subdirs) ; do make -C $$dir install || exit 1 ; done And now tell me how automake can make things simpler than this ;-) Hmm, I should have sent this as patch earlier - it would have been easier than updating the spec with tons of "make -C" calls... Regards, Christian Boltz [1] after dropping the large automake patch -- Eight Megabytes Always Continuously Swapping ;-) [Paketbeschreibung für emacs in SuSE 8.2] -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
Re: [apparmor] [PATCH] towards a common build infrastructure
Hello, Am Sonntag, 6. Mai 2012 schrieb Kees Cook: > On Sun, May 06, 2012 at 01:46:30AM +0200, Christian Boltz wrote: > > I prefer hand-written Makefiles - but that might be a matter of > > personal taste ;-) (and, in my case, missing knownledge about > > automake) > > I find it much easier to deal with large projects via autoconf and > automake. There is rarely anything 'strange' enough that it requires > much fiddling. I prefer it because it makes the build system familiar > to most people trying to understand it, and the templates are small. I'd guess even more people understand plain Makefiles ;-) and I can guarantee you that nearly nobody understands the automake-generated Makefiles (reading them might be needed to track down build issues) > > > === modified file 'README' > > > > You only deleted the lines describing the current build "system", > > but > > didn't add any note how to build AppArmor with automake... > > It's still there. It's just hugely reduced. (Which is why I wanted to > push for this.) Re-reading your patch a second time, I noticed that you are right. I expected some + lines in the README patch, but you found a clever way to recycle the existing lines ;-) > I just want the build system simplified. Well, I remember (from another project) that plain Makefiles can work with subdirs without problems. Something like this will work as Makefile in the top-level directory (untested braindump) subdirs=changehat/mod_apparmor changehat/pam_apparmor libraries/libapparmor parser profiles utils all: for dir in $(subdirs) ; do make -C $$dir || exit 1 ; done clean: for dir in $(subdirs) ; do make -C $$dir clean || exit 1 ; done install: for dir in $(subdirs) ; do make -C $$dir install || exit 1 ; done And now tell me how automake can make things simpler than this ;-) Hmm, I should have sent this as patch earlier - it would have been easier than updating the spec with tons of "make -C" calls... Regards, Christian Boltz [1] after dropping the large automake patch -- Eight Megabytes Always Continuously Swapping ;-) [Paketbeschreibung für emacs in SuSE 8.2] -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
Re: [apparmor] [PATCH] towards a common build infrastructure
Hi John, On Sat, May 05, 2012 at 05:33:23PM -0700, John Johansen wrote: > On 05/05/2012 02:38 PM, Kees Cook wrote: > > I'd like to start trying to move to a common autoconf infrastructure where > > the build elements are selected at configure time, etc. This will let us > > get rid of the unusual common/Make.rules logic, provide cleaner common > > build targets, potential for easier parallelization of the build, etc. > > > > This is the first step, which moves the configure.ac from > > libraries/libapparmor down into the top level directory. Currently > > only libraries/libapparmor is automake-ified, but with the autoconf > > infrastructure moved, we can start adding directories one at a time > > until all the by-hand Makefiles have been replaced. > > > > To help with with transition, I've added some by-hand Makefiles to > > needed subdirectories so that the top-level "make", "make check", and > > "make clean"s will get passed all the way down. > > > > Funny I had considered getting rid of autoconf completely for the unification. I'm strongly against dropping auto* when we're building so many libraries and bindings. All the hassle is kept far away. :) Take, for example, all the CPPFLAGS fixups I recently sent -- none of that would have been needed with a unified auto* setup. > At the moment lets just hold off on this, it isn't going to go into 2.8 so it > doesn't hurt waiting a little bit and we can discuss this next week. Sounds good. -Kees -- Kees Cook -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
Re: [apparmor] [PATCH] towards a common build infrastructure
Hi Christian, On Sun, May 06, 2012 at 01:46:30AM +0200, Christian Boltz wrote: > Am Samstag, 5. Mai 2012 schrieb Kees Cook: > > I'd like to start trying to move to a common autoconf infrastructure > > where the build elements are selected at configure time, etc. This > > will let us get rid of the unusual common/Make.rules logic, provide > > cleaner common build targets, potential for easier parallelization of > > the build, etc. > > I prefer hand-written Makefiles - but that might be a matter of personal > taste ;-) (and, in my case, missing knownledge about automake) I find it much easier to deal with large projects via autoconf and automake. There is rarely anything 'strange' enough that it requires much fiddling. I prefer it because it makes the build system familiar to most people trying to understand it, and the templates are small. > That said: Jeff maintained a patch to use automake everywhere up to > AppArmor 2.6.x. When I took over maintenance of the AppArmor package, > I dropped it in favor of the upstream build system. Even if I needed to > add 10 make calls from the spec, I still found that easier than > maintaining a 370 kB patch ;-) > > Nevertheless it might be easier for you to update Jeff's patch (it > doesn't apply to 2.7.x or trunk) than writing everything from scratch. > > If you are interested, you can find it on > https://build.opensuse.org/package/files?package=apparmor_2_6&project=security%3Aapparmor > >From there, take the files > - apparmor-2.5.1-unified-build (the main patch, 370 kB) > - apparmor-2.5.1-rpmlint-asprintf (small additional patch, depends on > the first patch, not sure if it is still needed) Ah, very cool. I'll take a look. > > === modified file 'README' > > You only deleted the lines describing the current build "system", but > didn't add any note how to build AppArmor with automake... It's still there. It's just hugely reduced. (Which is why I wanted to push for this.) $ sh ./autogen.sh $ sh ./configure --prefix=/usr --with-perl # see below $ make $ make check $ make install > Another question - what is the target version for changing the build > system? Do you want to include it in 2.8? 2.8 is too close. I'm in no rush; I just want the build system simplified. I'm happy to wait until 2.8 releases. -Kees -- Kees Cook -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
Re: [apparmor] [PATCH] towards a common build infrastructure
On 05/05/2012 02:38 PM, Kees Cook wrote: > I'd like to start trying to move to a common autoconf infrastructure where > the build elements are selected at configure time, etc. This will let us > get rid of the unusual common/Make.rules logic, provide cleaner common > build targets, potential for easier parallelization of the build, etc. > > This is the first step, which moves the configure.ac from > libraries/libapparmor down into the top level directory. Currently > only libraries/libapparmor is automake-ified, but with the autoconf > infrastructure moved, we can start adding directories one at a time > until all the by-hand Makefiles have been replaced. > > To help with with transition, I've added some by-hand Makefiles to > needed subdirectories so that the top-level "make", "make check", and > "make clean"s will get passed all the way down. > Funny I had considered getting rid of autoconf completely for the unification. At the moment lets just hold off on this, it isn't going to go into 2.8 so it doesn't hurt waiting a little bit and we can discuss this next week. john -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
Re: [apparmor] [PATCH] towards a common build infrastructure
Hello, Am Samstag, 5. Mai 2012 schrieb Kees Cook: > I'd like to start trying to move to a common autoconf infrastructure > where the build elements are selected at configure time, etc. This > will let us get rid of the unusual common/Make.rules logic, provide > cleaner common build targets, potential for easier parallelization of > the build, etc. I prefer hand-written Makefiles - but that might be a matter of personal taste ;-) (and, in my case, missing knownledge about automake) That said: Jeff maintained a patch to use automake everywhere up to AppArmor 2.6.x. When I took over maintenance of the AppArmor package, I dropped it in favor of the upstream build system. Even if I needed to add 10 make calls from the spec, I still found that easier than maintaining a 370 kB patch ;-) Nevertheless it might be easier for you to update Jeff's patch (it doesn't apply to 2.7.x or trunk) than writing everything from scratch. If you are interested, you can find it on https://build.opensuse.org/package/files?package=apparmor_2_6&project=security%3Aapparmor >From there, take the files - apparmor-2.5.1-unified-build (the main patch, 370 kB) - apparmor-2.5.1-rpmlint-asprintf (small additional patch, depends on the first patch, not sure if it is still needed) > === modified file 'README' You only deleted the lines describing the current build "system", but didn't add any note how to build AppArmor with automake... Another question - what is the target version for changing the build system? Do you want to include it in 2.8? Regards, Christian Boltz -- [Re: Wie krieg ich meinen Ratti in mutt zurueck?] In der procmail einfach angeben: formail -I From: ra...@gesindel.de (Ratti) Dann sind _ALLE_ Mails von Ratti. [Ratti (welcher denn? ;-) in suse-linux] -- AppArmor mailing list AppArmor@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
[apparmor] [PATCH] towards a common build infrastructure
I'd like to start trying to move to a common autoconf infrastructure where
the build elements are selected at configure time, etc. This will let us
get rid of the unusual common/Make.rules logic, provide cleaner common
build targets, potential for easier parallelization of the build, etc.
This is the first step, which moves the configure.ac from
libraries/libapparmor down into the top level directory. Currently
only libraries/libapparmor is automake-ified, but with the autoconf
infrastructure moved, we can start adding directories one at a time
until all the by-hand Makefiles have been replaced.
To help with with transition, I've added some by-hand Makefiles to
needed subdirectories so that the top-level "make", "make check", and
"make clean"s will get passed all the way down.
Signed-off-by: Kees Cook
=== modified file '.bzrignore'
--- .bzrignore 2011-08-09 08:17:42 +
+++ .bzrignore 2012-05-05 20:46:57 +
@@ -1,4 +1,18 @@
apparmor-*
+aclocal.m4
+autom4te.cache
+compile
+config.guess
+config.log
+config.status
+config.sub
+configure
+depcomp
+install-sh
+libtool
+ltmain.sh
+missing
+ylwrap
parser/po/*.mo
parser/af_names.h
parser/cap_names.h
@@ -25,23 +39,10 @@
parser/techdoc.log
parser/techdoc.pdf
parser/techdoc.toc
+profiles/common
libraries/libapparmor/Makefile
libraries/libapparmor/Makefile.in
-libraries/libapparmor/aclocal.m4
libraries/libapparmor/audit.log
-libraries/libapparmor/autom4te.cache
-libraries/libapparmor/compile
-libraries/libapparmor/config.guess
-libraries/libapparmor/config.log
-libraries/libapparmor/config.status
-libraries/libapparmor/config.sub
-libraries/libapparmor/configure
-libraries/libapparmor/depcomp
-libraries/libapparmor/install-sh
-libraries/libapparmor/libtool
-libraries/libapparmor/ltmain.sh
-libraries/libapparmor/missing
-libraries/libapparmor/ylwrap
libraries/libapparmor/doc/Makefile
libraries/libapparmor/doc/Makefile.in
libraries/libapparmor/doc/*.2
=== modified file 'Makefile'
--- Makefile2012-03-22 20:17:48 +
+++ Makefile2012-05-05 21:14:28 +
@@ -2,15 +2,17 @@
#
OVERRIDE_TARBALL=yes
+all: build
+
include common/Make.rules
-DIRS=parser \
- profiles \
- utils \
- libraries/libapparmor \
- changehat/mod_apparmor \
- changehat/pam_apparmor \
- tests
+SUBDIRS=\
+ libraries \
+ utils \
+ parser \
+ changehat \
+ profiles \
+ tests
REPO_URL?=lp:apparmor
# alternate possibilities to export from
@@ -25,6 +27,24 @@
# embedded in ${VERSION}
TAG_VERSION=$(subst ~,-,${VERSION})
+# Support the top-level automake targets during transition.
+.PHONY: am--refresh
+ for dir in libraries/libapparmor; do \
+ make -C $$dir am--refresh || exit 1; \
+ done
+
+.PHONY: build
+build: configure
+ for dir in $(SUBDIRS); do \
+ make -C $$dir || exit 1; \
+ done
+
+.PHONY: check
+check:
+ for dir in $(SUBDIRS); do \
+ make -C $$dir check || exit 1; \
+ done
+
.PHONY: tarball
tarball: clean
REPO_VERSION=`$(value REPO_VERSION_CMD)` ; \
@@ -50,15 +70,14 @@
.PHONY: clean
clean:
-rm -rf ${RELEASE_DIR} ./apparmor-${VERSION}~*
- for dir in $(DIRS); do \
+ for dir in $(SUBDIRS); do \
make -C $$dir clean; \
done
.PHONY: setup
setup:
- cd $(__SETUP_DIR)/libraries/libapparmor && ./autogen.sh
+ cd $(__SETUP_DIR) && ./autogen.sh
.PHONY: tag
tag:
bzr tag apparmor_${TAG_VERSION}
-
=== modified file 'README'
--- README 2011-09-12 20:30:17 +
+++ README 2012-05-05 20:48:29 +
@@ -58,9 +58,6 @@
To build and install AppArmor userspace on your system, build and install in
the following order.
-
-libapparmor:
-$ cd ./libraries/libapparmor
$ sh ./autogen.sh
$ sh ./configure --prefix=/usr --with-perl # see below
$ make
@@ -71,40 +68,6 @@
and --with-ruby, to generate python and ruby bindings to libapparmor,
respectively.]
-
-Utilities:
-$ cd utils
-$ make
-$ make check
-$ make install
-
-
-parser:
-$ cd parser
-$ make
-$ make check
-$ make install
-
-
-Apache mod_apparmor:
-$ cd changehat/mod_apparmor
-$ make # depends on libapparmor having been built first
-$ make install
-
-
-PAM AppArmor:
-$ cd changehat/pam_apparmor
-$ make # depends on libapparmor having been built first
-$ make install
-
-
-Profiles:
-$ cd profiles
-$ make
-$ make check # depends on the parser having been built first
-$ make install
-
-
[Note that for the parser and the utils, if you only with to build/use
some of the locale languages, you can override the default by passing
the LANGS arguments to make; e.g. make all install "LANGS=en_US fr".]
@@ -183,4 +146,3 @@
---
TODO
-
=== renamed file 'libraries/libapparmor/autogen.sh' => 'autogen.sh'
=== added file 'changehat/Makefile'
--- changehat/Makefile 1970-01-01 00:00:00 +
+++ changehat/Makefile 2012-05-05 21:14:51 +
