Bug#549407: [buildd-tools-devel] Bug#549407: ivtools 1.2.6-1 FTBFS on sparc and powerpc
clone 549407 -1 reassign -1 xutils-dev retitle -1 imake: Must not unconditionally mangle pre-defined symbols in paths. thanks On Tue, Dec 08, 2009 at 03:01:12PM +, Roger Leigh wrote: Thanks for identifying the cause of the problem! So, to state the problem clearly: Imake is substituting the $(ARCH) part of the path to something else. Such as 'i386' being swapped for something else entirely, thus resulting in an invalid path. Exactly. This is particularly obscure, since not all arch names have the associated symbol defined, e.g., I could not reproduce this with amd64, and there were no problems with other autobuilders using new sbuild, but there were with i386, sparc and powerpc. This is, IMO, completely broken on the part of Imake. Agreed I'm reluctant to alter sbuild to accommodate such bad behaviour. For one thing, it can substitute /any part/ of the path, so there's no guarantee it won't randomly break on the XX random part or any other path component for any given build. The fix just makes the arch mismatch because underscore makes the two parts a single token, but that is not to say it will /never/ match. I accept that it solves the immediate issue, but it doesn't correct the fundamental underlying defect in imake. What's worse is that the random path might actually be /valid/, in which case it might scribble junk into, or delete files from, a directory other than the build directory. Unlikely, but possible, so a potential security problem on the buildd. Agreed also, but note that there is the same potential problem with the current setup. IIRC, few packages still use imake, so this is not at all a generalized problem. The only action I'd currently expect to be considered from the sbuild side is documenting this misbehavior. I agree that this is an imake misbehavior, so I am cloning this bug report and reassigning the clone to xutils-dev. I am keeping current RC severity, xutils-dev maintainers, please readjust at your will. Is this possible to fix in ivtools using the -u option to undefine things as suggested in the FAQ? Given the package-specific nature of the problem, I feel this would be a more appropriate place for a fix. Yes, that is the scheduled fix. Unfortunately, the buildd-tools mailing list stripped the patch, this is the relevant part, +# Make sure this symbol is disabled when imake is invoked. +MAKE += SPECIAL_IMAKEFLAGS=-U$(ARCH) + and changed make-$(MAKE). In practice, this should only affect the Makefile and Makefiles targets. However, I did not check carefully if there is other indirect imake invocation, so doing things this way does not hurt. I expect to upload a fixed ivtools NMU today. Cheers, -- Agustin -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#549407: [buildd-tools-devel] Bug#549407: ivtools 1.2.6-1 FTBFS on sparc and powerpc
On Wed, Dec 09, 2009 at 01:55:11PM +0100, Agustin Martin wrote: I expect to upload a fixed ivtools NMU today. Uploaded. Took a bit longer because of a new RC bug with current sid gcc 4.4 and another problem with uintptr_t. Patch is attached. Unfortunately, seems that although it properly built in i386 pbuilder, at least in amd64 there are more gcc4.4 problems pending that did not show in i386. -- Agustin diff -u ivtools-1.2.6/debian/changelog ivtools-1.2.6/debian/changelog --- ivtools-1.2.6/debian/changelog +++ ivtools-1.2.6/debian/changelog @@ -1,3 +1,20 @@ +ivtools (1.2.6-1+nmu3) unstable; urgency=low + + * Non-maintainer upload. + * debian/rules: +- Reverting 1.2.6-1+nmu2 -j1 addition to keep package clean. +- Fix ARCH evaluation. +- Use generic $(MAKE) instead of make. +- Make sure imake is always called with -U$(ARCH) (Closes: #549407). + * 01_configure.diff: Handle disabling of early binary stripping. + * 015_Makefile_dontstrip.diff: Disable patch. + * 49_string.c_constchar.diff: New patch to deal with gcc 4.4 'invalid +conversion from 'const char*' to 'char*'' errors (Closes: #560203). + * 48_gcc40_int_pointer_errors.diff: Modified to use (unsigned intptr_t) +instead of (uintptr_t) to avoid out-of-context errors. + + -- Agustin Martin Domingo agmar...@debian.org Wed, 09 Dec 2009 18:13:03 +0100 + ivtools (1.2.6-1+nmu2) unstable; urgency=low * Non-maintainer upload. diff -u ivtools-1.2.6/debian/rules ivtools-1.2.6/debian/rules --- ivtools-1.2.6/debian/rules +++ ivtools-1.2.6/debian/rules @@ -16,7 +16,7 @@ # so for alpha we just don't have ace, who uses it # anyhow ? -ARCH = $(dpkg --print-architecture) +ARCH = $(shell dpkg --print-architecture) ifeq $(ARCH) alpha ACE = @@ -24,6 +24,9 @@ # ACE = --with-ace=/usr/include --with-ace-libs=/usr/lib endif +# Make sure this symbol is disabled when imake is invoked. See #549407. +MAKE += SPECIAL_IMAKEFLAGS=-U$(ARCH) + build: build-stamp build-stamp: dh_testdir @@ -41,9 +44,9 @@ # Compile the packages - make Makefile ARCH=LINUX - make Makefiles ARCH=LINUX - make -j1 ARCH=LINUX + $(MAKE) Makefile ARCH=LINUX + $(MAKE) Makefiles ARCH=LINUX + $(MAKE) ARCH=LINUX # BUILD FINISHED # --- @@ -59,8 +62,8 @@ --mandir=$(CURDIR)/debian/tmp/usr/share/man cd src/scripts \ - make ARCH=LINUX clean \ - make ARCH=LINUX CONFIGDIRSPEC='-T template -I/usr/lib/ivtools/config -DCURDIR=\$(CURDIR)'\ + $(MAKE) ARCH=LINUX clean \ + $(MAKE) ARCH=LINUX CONFIGDIRSPEC='-T template -I/usr/lib/ivtools/config -DCURDIR=\$(CURDIR)'\ MAKEMAKESPEC='ARCH=LINUX' touch build-stamp @@ -83,8 +86,8 @@ -rm config.cache -rm -rf static shared - [ ! -f Makefile ] || make ARCH=LINUX clean - [ ! -f src/scripts/Makefile ] || (cd src/scripts;make clean) + [ ! -f Makefile ] || $(MAKE) ARCH=LINUX clean + [ ! -f src/scripts/Makefile ] || (cd src/scripts;$(MAKE) clean) # remove backups -rm -f `find . -name *~` @@ -102,10 +105,10 @@ dh_installdirs # Add here commands to install the package into debian/tmp. - make install ARCH=LINUX DESTDIR=$(CURDIR)/debian/tmp + $(MAKE) install ARCH=LINUX DESTDIR=$(CURDIR)/debian/tmp cp debian/template $(CURDIR)/debian/tmp/usr/lib/ivtools/config/ - [ ! -f src/glyphs/Makefile ] || (cd src/glyphs make clean) + [ ! -f src/glyphs/Makefile ] || (cd src/glyphs $(MAKE) clean) # don't include the scripts # ivtools installs the libACE link, we remove it ... hack diff -u ivtools-1.2.6/debian/patches/series ivtools-1.2.6/debian/patches/series --- ivtools-1.2.6/debian/patches/series +++ ivtools-1.2.6/debian/patches/series @@ -1,5 +1,4 @@ 01_configure.diff -015_Makefile_dontstrip.diff 02_rules_def.diff 025_params_def.diff 026_geomobjs_c.diff @@ -42,0 +42 @@ +49_string.c_constchar.diff diff -u ivtools-1.2.6/debian/patches/01_configure.diff ivtools-1.2.6/debian/patches/01_configure.diff --- ivtools-1.2.6/debian/patches/01_configure.diff +++ ivtools-1.2.6/debian/patches/01_configure.diff @@ -1,8 +1,15 @@ -Index: ivtools-1.2/configure +This patch modfies the configure scripts for different purposes: + +* Modify how config/config.mk is built: + - Make 'make CPU' more silent. + - Do not strip binaries during the build process. Will be done later +from debian/rules. + +Index: ivtools-1.2.6/configure === ivtools-1.2.orig/configure 2009-09-04 12:19:47.0 -0400 -+++ ivtools-1.2/configure 2009-09-04 12:20:02.0 -0400 -@@ -5129,8 +5129,8 @@ +--- ivtools-1.2.6.orig/configure 2009-11-30 12:08:21.0 +0100 ivtools-1.2.6/configure 2009-11-30 12:08:54.0 +0100 +@@ -5129,8 +5129,10 @@ echo XCONFIGDIR = $x_libraries/X11/config config/config.mk echo writing \ABSTOP = `pwd`\ echo ABSTOP = `pwd` config/config.mk @@ -10,14 +17,16 @@ -echo CPU = `make CPU` config/config.mk +echo writing \CPU = `make -s CPU`\
Bug#549407: [buildd-tools-devel] Bug#549407: ivtools 1.2.6-1 FTBFS on sparc and powerpc
On Tue, Dec 08, 2009 at 03:11:32AM +0100, Agustin Martin wrote:
2009/12/6 Agustin Martin agmar...@debian.org:
Good news. I finally found the reason for this problem. An explanation
was not that far. Quoting http://www.ivtools.org/ivtools/faq.html,
---
... For example, most
PC-based uses of gcc have i386 defined to 1, so a path like
/usr/src/i386/ivtools-1.0 gets expanded to /usr/src/1/ivtools-1.0. ..
---
As I understand it, this problem should also be present in other
packages using imake. Not sure if many packages still use imake, but
if so there is a problem with the temporary dir naming in sbuild.
In case this is ever needed, seems that is enough to change to a
lowbar one of the hyphen $(ARCH) boundaries in tempdir name, like in
-
--- Build.pm.orig 2009-11-20 19:48:42.0 +0100
+++ Build.pm2009-12-07 01:00:07.0 +0100
@@ -233,7 +233,7 @@
$self-set('Chroot Build Dir',
tempdir($self-get_conf('USERNAME') . '-' .
$self-get('Package_SVersion') . '-' .
- $self-get('Arch') . '-XX',
+ $self-get('Arch') . '_XX',
DIR = $session-get('Build Location')));
# TODO: Don't hack the build location in; add a means to customise
# the chroot directly.
Thanks for identifying the cause of the problem!
So, to state the problem clearly: Imake is substituting the
$(ARCH) part of the path to something else. Such as 'i386' being
swapped for something else entirely, thus resulting in an
invalid path.
This is, IMO, completely broken on the part of Imake. I'm reluctant
to alter sbuild to accommodate such bad behaviour. For one thing,
it can substitute /any part/ of the path, so there's no guarantee it
won't randomly break on the XX random part or any other path
component for any given build. The fix just makes the arch
mismatch because underscore makes the two parts a single token, but
that is not to say it will /never/ match. I accept that it solves
the immediate issue, but it doesn't correct the fundamental
underlying defect in imake.
What's worse is that the random path might actually be /valid/, in
which case it might scribble junk into, or delete files from, a
directory other than the build directory. Unlikely, but possible,
so a potential security problem on the buildd.
Is this possible to fix in ivtools using the -u option to undefine
things as suggested in the FAQ? Given the package-specific nature
of the problem, I feel this would be a more appropriate place for
a fix.
Regards,
Roger
--
.''`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`-GPG Public Key: 0x25BFB848 Please GPG sign your mail.
signature.asc
Description: Digital signature
Bug#549407: [buildd-tools-devel] Bug#549407: ivtools 1.2.6-1 FTBFS on sparc and powerpc
2009/12/6 Agustin Martin agmar...@debian.org:
Good news. I finally found the reason for this problem. An explanation
was not that far. Quoting http://www.ivtools.org/ivtools/faq.html,
---
... For example, most
PC-based uses of gcc have i386 defined to 1, so a path like
/usr/src/i386/ivtools-1.0 gets expanded to /usr/src/1/ivtools-1.0. ..
---
As I understand it, this problem should also be present in other
packages using imake. Not sure if many packages still use imake, but
if so there is a problem with the temporary dir naming in sbuild.
In case this is ever needed, seems that is enough to change to a
lowbar one of the hyphen $(ARCH) boundaries in tempdir name, like in
-
--- Build.pm.orig 2009-11-20 19:48:42.0 +0100
+++ Build.pm2009-12-07 01:00:07.0 +0100
@@ -233,7 +233,7 @@
$self-set('Chroot Build Dir',
tempdir($self-get_conf('USERNAME') . '-' .
$self-get('Package_SVersion') . '-' .
- $self-get('Arch') . '-XX',
+ $self-get('Arch') . '_XX',
DIR = $session-get('Build Location')));
# TODO: Don't hack the build location in; add a means to customise
# the chroot directly.
Cheers,
--
Agustin
--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#549407: [buildd-tools-devel] Bug#549407: ivtools 1.2.6-1 FTBFS on sparc and powerpc
2009/11/9 Agustin Martin agmar...@debian.org: Answering myself. No, it is not. I have prepared an sbuild+schroot setup in a i386 box, using lenny schroot and bleeding edge sbuild (HEAD from sbuild git repo, 355a4dffe742445713ea089436da077fc7a40ae8). I can reproduce the problem with that sbuild version in the i386 box. However, I cannot reproduce it at all with pbuilder or with plain build in normal filesystem. ivtools uses imake to regenerate makefiles on each run. When the problem is reproduced, I noticed that the path set by imake is something (wrong) like .../buildd-ivtools_1.2.6-1+nmu2-1-2p04ki while the real path is something like .../buildd-ivtools_1.2.6-1+nmu2-powerpc-2p04ki Good news. I finally found the reason for this problem. An explanation was not that far. Quoting http://www.ivtools.org/ivtools/faq.html, --- * The absolute pathname used by the make depend pass is not the same as the ABSTOP pathname written to config/config.mk by the configure script. There is a vulnerability in the make Makefiles pass (the use of imake), where absolute pathnames that incorporate predefined C symbols get these symbols expanded to another value. For example, most PC-based uses of gcc have i386 defined to 1, so a path like /usr/src/i386/ivtools-1.0 gets expanded to /usr/src/1/ivtools-1.0. To check the set of pre-defined symbols, enter this command: touch test.c;gcc -dM -E test.c. You can either rename the affected directory in which ivtools resides, or you can add -U undefine arguments as necessary to the imake command line. See the definition of ImakeFlags in config/site.def.NETBSD for an example. --- As I understand it, this problem should also be present in other packages using imake. Not sure if many packages still use imake, but if so there is a problem with the temporary dir naming in sbuild. However, this problem seems a bit nonsensic, so if only few packages still use imake we can try working around this problem by making sure imake is invoked with -U`dpkg --print-architecture`. I hope this has no side effects. I am attaching a patch with the current changes in my box. I can successfully build ivtools with sbuild in this i386 box where I previously could not. It includes some other minor changes: - Revert the -j introduced in my last NMU. - Disable 015_Makefile_dontstrip.diff. Since imake rewrites Makefile, patches for it cannot be reverted later if built in the same dir. This is now done in configure through config.mk. Comments are more than welcome. If there are no objections I will set the final version number and upload at some time in the middle of next week. Cheers, -- Agustin diff -u ivtools-1.2.6/debian/changelog ivtools-1.2.6/debian/changelog --- ivtools-1.2.6/debian/changelog +++ ivtools-1.2.6/debian/changelog @@ -1,3 +1,16 @@ +ivtools (1.2.6-1+nmu2~a3) unstable; urgency=low + + * Non-maintainer upload. + * debian/rules: +- Reverting 1.2.6-1+nmu2 -j1 addition to keep package clean. +- Fix ARCH evaluation. +- Use generic $(MAKE) instead of make. +- Make sure imake is always called with -U$(ARCH) (Closes: #549407). + * 01_configure.diff: Handle disabling of early binary stripping. + * 015_Makefile_dontstrip.diff: Disable patch. + + -- Agustin Martin Domingo agmar...@debian.org Sun, 06 Dec 2009 04:41:51 +0100 + ivtools (1.2.6-1+nmu2) unstable; urgency=low * Non-maintainer upload. diff -u ivtools-1.2.6/debian/rules ivtools-1.2.6/debian/rules --- ivtools-1.2.6/debian/rules +++ ivtools-1.2.6/debian/rules @@ -16,7 +16,7 @@ # so for alpha we just don't have ace, who uses it # anyhow ? -ARCH = $(dpkg --print-architecture) +ARCH = $(shell dpkg --print-architecture) ifeq $(ARCH) alpha ACE = @@ -24,6 +24,9 @@ # ACE = --with-ace=/usr/include --with-ace-libs=/usr/lib endif +# Make sure this symbol is disabled when imake is invoked. +MAKE += SPECIAL_IMAKEFLAGS=-U$(ARCH) + build: build-stamp build-stamp: dh_testdir @@ -41,9 +44,9 @@ # Compile the packages - make Makefile ARCH=LINUX - make Makefiles ARCH=LINUX - make -j1 ARCH=LINUX + $(MAKE) Makefile ARCH=LINUX + $(MAKE) Makefiles ARCH=LINUX + $(MAKE) ARCH=LINUX # BUILD FINISHED # --- @@ -59,8 +62,8 @@ --mandir=$(CURDIR)/debian/tmp/usr/share/man cd src/scripts \ - make ARCH=LINUX clean \ - make ARCH=LINUX CONFIGDIRSPEC='-T template -I/usr/lib/ivtools/config -DCURDIR=\$(CURDIR)'\ + $(MAKE) ARCH=LINUX clean \ + $(MAKE) ARCH=LINUX CONFIGDIRSPEC='-T template -I/usr/lib/ivtools/config -DCURDIR=\$(CURDIR)'\ MAKEMAKESPEC='ARCH=LINUX' touch build-stamp @@ -83,8 +86,8 @@ -rm config.cache -rm -rf static shared - [ ! -f Makefile ] || make ARCH=LINUX clean - [ ! -f src/scripts/Makefile ] || (cd src/scripts;make clean) + [ ! -f Makefile ] || $(MAKE) ARCH=LINUX clean + [ ! -f src/scripts/Makefile ] || (cd src/scripts;$(MAKE) clean) # remove
Bug#549407: [buildd-tools-devel] Bug#549407: ivtools 1.2.6-1 FTBFS on sparc and powerpc
On Mon, Nov 09, 2009 at 01:20:55PM +0100, Agustin Martin wrote: reassign 549407 buildd.debian.org,ivtools thanks 2009/11/6 Roger Leigh rle...@codelibre.net: On Fri, Nov 06, 2009 at 04:20:31PM +0100, Agustin Martin wrote: I am looking at #549407 and noticed that both powerpc and sparc autobuilders seem to use two temporary dirs at the same time, I think those buildds may be missing the binNMU versioning fix in commit 817aea8c, but I don't have access to check that. Thanks for the hint. May that be the origin for #549407? Answering myself. No, it is not. I have prepared an sbuild+schroot setup in a i386 box, using lenny schroot and bleeding edge sbuild (HEAD from sbuild git repo, 355a4dffe742445713ea089436da077fc7a40ae8). I can reproduce the problem with that sbuild version in the i386 box. However, I cannot reproduce it at all with pbuilder or with plain build in normal filesystem. ivtools uses imake to regenerate makefiles on each run. When the problem is reproduced, I noticed that the path set by imake is something (wrong) like .../buildd-ivtools_1.2.6-1+nmu2-1-2p04ki while the real path is something like .../buildd-ivtools_1.2.6-1+nmu2-powerpc-2p04ki I tried to look how that happens, or at least find a workaround, but was still unlucky. Why this does not happen in all autobuilders using new sbuild is puzzling me. More updates to this bug report. ivtools now builds in powerpc. It has now been queued to voltaire (using old sbuild) instead of praetorius (using the new one). The only arch where is not building is sparc, where lebrun has been used for all failed builds. Build *suceeds* in amd64 (sbuild/amd64 98 on nautilus), ia64 (sbuild/ia64 98 on caballero), mips (sbuild/mips 99.99 on ball), having what seems to be an old sbuild. Build *suceeds* in alpha (sbuild 0.58.2 (15 Jun 2009) on goetzs), armel (sbuild 0.58.2 (15 Jun 2009) on argento), mipsel ( sbuild 0.58.2 (15 Jun 2009) on rem), s390 (sbuild 0.58.2 (31 Jul 2009) on debian-31), having the new sbuild. Build *fails* in powerpc (sbuild 0.58.2 (15 Jun 2009) on praetorius), sparc (sbuild 0.58.2 (15 Jun 2009) on lebrun), all having the new sbuild. I wonder what is different between buildds using new sbuild that makes this work in ones and fail in others. -- Agustin -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#549407: [buildd-tools-devel] Bug#549407: ivtools 1.2.6-1 FTBFS on sparc and powerpc
reassign 549407 buildd.debian.org,ivtools thanks 2009/11/6 Roger Leigh rle...@codelibre.net: On Fri, Nov 06, 2009 at 04:20:31PM +0100, Agustin Martin wrote: I am looking at #549407 and noticed that both powerpc and sparc autobuilders seem to use two temporary dirs at the same time, I think those buildds may be missing the binNMU versioning fix in commit 817aea8c, but I don't have access to check that. Thanks for the hint. May that be the origin for #549407? I have looked at all the ivtools logs and everything points to powerpc (praetorious) and sparc (lebrun) autobuilders having something different to all the other similar autobuilders where ivtools has been processed. Build *suceeds* in amd64 (sbuild/amd64 98 on nautilus), ia64 (sbuild/ia64 98 on caballero), mips (sbuild/mips 99.99 on ball), having what seems to be an old sbuild. Build *suceeds* in alpha (sbuild 0.58.2 (15 Jun 2009) on goetzs), armel (sbuild 0.58.2 (15 Jun 2009) on argento), mipsel ( sbuild 0.58.2 (15 Jun 2009) on rem), s390 (sbuild 0.58.2 (31 Jul 2009) on debian-31), having the new sbuild. Build *fails* in powerpc (sbuild 0.58.2 (15 Jun 2009) on praetorius), sparc (sbuild 0.58.2 (15 Jun 2009) on lebrun), all having the new sbuild. Is only in this last two autobuilders where I see two different temporaty directories being used simultaneously. buildd-ivtools_1.2.6-1+nmu2-1-2p04ki != buildd-ivtools_1.2.6-1+nmu2-powerpc-2p04ki buildd-ivtools_1.2.6-1+nmu2-1-6yM9Gd != buildd-ivtools_1.2.6-1+nmu2-sparc-6yM9G In all of alpha, armel, mipsel and s390 autobuilders, where build suceeds, no such behavior is found and a single temporary dir is used consistently during all the build. SInce there is something strange in powerpc and sparc autobuilders that may be related to #549407 I am reassigning this bug report to both buildd.debian.org and ivtools. If this is finally not related to the autobuilders, please reassign it back to ivtools alone. I have a new upload ready just to remove the -j1 stuff I added in previous upload. I was convinced at that time that this was a race condition where -j1 may help. Since this seems not true, better leave the package wihout the -j1, so changes are kept minimal and parallel processing can be attempted. I however will not upload this shortly. Will wait before for buildd admins comments. Thanks for your help, Regards, -- Agustin -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
