Bug#671074: dpkg-buildpackage: option for building multiple times in a row

2012-07-02 Thread Jakub Wilk 

* Jonathan Nieder jrnie...@gmail.com, 2012-05-01, 13:22:
The attached patch adds option to build package multiple times in a 
row. This is useful for testing correctness of the clean target.


The patch looks noisy, because I had to indent a few dozen of lines; 
but other than that, it adds just a few lines.


If case you wonder why can't I just call dpkg-buildpackage in a loop: 
I can't if run it through a wrapper like sbuild or $VCS-buildpackage.


Neat --- sounds like a useful test.

I'm not convinced this is worth a dpkg-buildpackage option on its own.


Well, I am.

Wouldn't other sequences of targets be worth testing, too? (E.g., 
debian/rules build  debian/rules clean  fakeroot 
debian/rules binary.)


I don't think so. (Testing if binary without build works is possibly 
useful, but that's already doable.)


I assume in the $VCS-buildpackage case you are building with the 
--export-dir option so the build tree is wiped out between builds.


I've never heard of -export-dir.


Possible alternative methods:

a) teaching sbuild and $VCS-buildpackage an option to use another 
command instead of dpkg-buildpackage, or


b) (as a hack, if (a) is too fussy) overriding dpkg-buildpackage with 
a wrapper that appears earlier on the $PATH.


I'm not interested in patching multiple builders or dirtying my build 
environment.


--
Jakub Wilk



--
To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#671074: dpkg-buildpackage: option for building multiple times in a row

2012-07-02 Thread Jonathan Nieder 
Hi again,

Jakub Wilk wrote:
 * Jonathan Nieder jrnie...@gmail.com, 2012-05-01, 13:22:

I'm not convinced this is worth a dpkg-buildpackage option on its own.

 Well, I am.

Good, now our positions are staked out. Hopefully it is possible to
take care of all our needs without interfering with the others.

I am concerned that this --build-twice option would clutter the
documentation and make it harder for readers to understand the role
dpkg-buildpackage is supposed to have (that is, it is a layering
violation). Worse, it seems to be designed to work around a bug in
tools that call dpkg-buildpackage --- instead of providing a
convenient interface to run some other sequence of commands instead of
dpkg-buildpackage, if I understand correctly you are saying they
only allow passing options to it. So if someone wants to test what
happens when building a package three times in a row, say, (or when
building for a second, killing the build, cleaning up, and then
finishing the build) there would need to be yet another commandline
option.

 Possible alternative methods:

 a) teaching sbuild and $VCS-buildpackage an option to use another command
 instead of dpkg-buildpackage, or

 b) (as a hack, if (a) is too fussy) overriding dpkg-buildpackage with a
 wrapper that appears earlier on the $PATH.

 I'm not interested in patching multiple builders or dirtying my build
 environment.

In case (a), wouldn't you only need to patch one builder (namely the
one you use), and wouldn't case (b) be possible without dirtying your
build environment if another package (or a pbuilder hook, or whatever)
provided the dpkg-buildpackage wrapper?

I can imagine there being other reasons to want to implement this in
dpkg (are there some?), but without such a reason I am concerned that
this is expanding the scope of dpkg-buildpackage --- instead of
supporting the general case with sufficient generality, supporting
special cases.

Hoping that clarifies,
Jonathan



-- 
To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#671074: dpkg-buildpackage: option for building multiple times in a row

2012-07-02 Thread Jakub Wilk 

* Jonathan Nieder jrnie...@gmail.com, 2012-07-02, 09:24:

Possible alternative methods:

a) teaching sbuild and $VCS-buildpackage an option to use another 
command instead of dpkg-buildpackage, or


b) (as a hack, if (a) is too fussy) overriding dpkg-buildpackage with 
a wrapper that appears earlier on the $PATH.


I'm not interested in patching multiple builders or dirtying my build 
environment.


In case (a), wouldn't you only need to patch one builder (namely the 
one you use),


Is this a polite way of saying nobody else but you would use this 
feature anyway?


In any case, arguing other this is not a productive use of my time. 
dpkg developers: if you are are not interested in the patch I proposed, 
please tell me as soon as possible, so that I can close the bug and work 
on another (inferior) solution to my problem.


--
Jakub Wilk



--
To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#671074: dpkg-buildpackage: option for building multiple times in a row

2012-07-02 Thread Jonathan Nieder 
Jakub Wilk wrote:
 * Jonathan Nieder jrnie...@gmail.com, 2012-07-02, 09:24:

 In case (a), wouldn't you only need to patch one builder (namely
 the one you use),

 Is this a polite way of saying nobody else but you would use this
 feature anyway?

No.  I said before that the proposed feature would be very useful and
I just think that dpkg-buildpackage is the wrong place to implement
it.  Clearly I am not communicating well at all.

(Similarly, resolving build-time dependencies, finding an appropriate
repository to download them from, and installing them in the
appropriate order is very useful.  There is more than one non-dpkg
codebase implementing that functionality, and I don't think that's a
terrible thing.  However, each user did not need to implement it
multiple times.)

Anyway, it's true that these are only my own reactions, and I don't
speak for anyone else.  If it turns out that my explanations did not
reveal anything more useful than an opinion, and not, say, pointers
for parts that could be documented better, then I'm sorry about that.



-- 
To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#671074: dpkg-buildpackage: option for building multiple times in a row

2012-05-01 Thread Jakub Wilk 

Package: dpkg-dev
Version: 1.16.3
Severity: wishlist
Tags: patch
User: d...@packages.debian.org
Usertags: dpkg-buildpackage

The attached patch adds option to build package multiple times in a row. 
This is useful for testing correctness of the clean target.


The patch looks noisy, because I had to indent a few dozen of lines; but 
other than that, it adds just a few lines.


If case you wonder why can't I just call dpkg-buildpackage in a loop: I 
can't if run it through a wrapper like sbuild or $VCS-buildpackage.


--
Jakub Wilk
diff --git a/man/dpkg-buildpackage.1 b/man/dpkg-buildpackage.1
--- a/man/dpkg-buildpackage.1
+++ b/man/dpkg-buildpackage.1
@@ -68,6 +68,9 @@
 .IP \fB9.\fP 3
 If \fB\-tc\fP is specified, it will call \fBfakeroot debian/rules clean\fP
 again. Finally it calls \fBdpkg\-source \-\-after\-build\fP.
+.P
+If \fB\-l\fP\fIN\fP is specified, steps 1-9 are repeated \fIN\fP times, with
+the exception that steps 6 and 8 are only executed in the last iteration.
 .
 .SH OPTIONS
 .TP
@@ -90,6 +93,10 @@
 .B \-F
 Specifies a normal full build, binary and source packages will be built.
 This is the same as the default case when no build option is specified.
+.TP
+.BI \-l N
+Build the package \fIN\fP times in a row.  This is useful for testing the
+clean target.
 .P
 .BI \-\-target= target
 .br
diff --git a/scripts/dpkg-buildpackage.pl b/scripts/dpkg-buildpackage.pl
--- a/scripts/dpkg-buildpackage.pl
+++ b/scripts/dpkg-buildpackage.pl
@@ -73,6 +73,7 @@
   -A binary-only, only arch-indep files. }
   -S source only, no binary files. }
   -F normal full build (binaries and sources).
+  -ln  build n times in a row
   -tsystem set GNU system type.   } passed to dpkg-architecture
   -vversionchanges since version version.  }
   -mmaint  maintainer for package is maint.}
@@ -123,6 +124,7 @@
 my $targetarch = my $targetgnusystem = '';
 my $call_target = '';
 my $call_target_as_root = 0;
+my $nloops = 1;
 my (@checkbuilddep_opts, @changes_opts, @source_opts);
 
 use constant BUILD_DEFAULT= 1;
@@ -229,6 +231,8 @@
 	!build_normal  usageerr(_g(cannot combine %s and %s), $_, build_opt);
 	$include = BUILD_ALL;
 	@checkbuilddep_opts = ();
+} elsif (/^-l(\d+)$/) {
+	$nloops = int($1);
 } elsif (/^-v(.*)$/) {
 	$since = $1;
 } elsif (/^-m(.*)$/) {
@@ -352,162 +356,172 @@
 chmod(0755, debian/rules); # No checks of failures, non fatal
 }
 
-unless ($call_target) {
+for my $iter (1 .. $nloops) {
+
+unless ($call_target) {
+	chdir('..') or syserr('chdir ..');
+	withecho('dpkg-source', @source_opts, '--before-build', $dir);
+	chdir($dir) or syserr(chdir $dir);
+}
+
+if ($checkbuilddep) {
+	if ($admindir) {
+	push @checkbuilddep_opts, --admindir=$admindir;
+	}
+
+	system('dpkg-checkbuilddeps', @checkbuilddep_opts);
+	if (not WIFEXITED($?)) {
+	subprocerr('dpkg-checkbuilddeps');
+	} elsif (WEXITSTATUS($?)) {
+	warning(_g(Build dependencies/conflicts unsatisfied; aborting.));
+	warning(_g((Use -d flag to override.)));
+
+	if (build_sourceonly) {
+		warning(_g(This is currently a non-fatal warning with -S, but));
+		warning(_g(will probably become fatal in the future.));
+	} else {
+		exit 3;
+	}
+	}
+}
+
+if ($call_target) {
+	if ($call_target_as_root or
+	$call_target =~ /^(clean|binary(|-arch|-indep))$/)
+	{
+	withecho(@rootcommand, @debian_rules, $call_target);
+	} else {
+	withecho(@debian_rules, $call_target);
+	}
+	exit 0;
+}
+
+unless ($noclean) {
+	withecho(@rootcommand, @debian_rules, 'clean');
+}
+unless (build_binaryonly) {
+	warning(_g(it is a bad idea to generate a source package  .
+		   without cleaning up first, it might contain undesired  .
+		   files.)) if $noclean;
+	chdir('..') or syserr('chdir ..');
+	withecho('dpkg-source', @source_opts, '-b', $dir);
+	chdir($dir) or syserr(chdir $dir);
+}
+
+unless ($buildtarget eq build or scalar(@debian_rules)  1) {
+	# Verify that build-{arch,indep} are supported. If not, fallback to build.
+	# This is a temporary measure to not break too many packages on a flag day.
+	my $pid = spawn(exec = [ make, -f, @debian_rules, -qn, $buildtarget ],
+			from_file = /dev/null, to_file = /dev/null,
+			error_to_file = /dev/null);
+	my $cmdline = make -f @debian_rules -qn $buildtarget;
+	wait_child($pid, nocheck = 1, cmdline = $cmdline);
+	my $exitcode = WEXITSTATUS($?);
+	subprocerr($cmdline) unless WIFEXITED($?);
+	if ($exitcode == 2) {
+	warning(_g(%s must be updated to support the 'build-arch' and  .
+		   'build-indep' targets (at least '%s' seems to be  .
+		   missing)), @debian_rules, $buildtarget);
+	$buildtarget = build;
+	}
+}
+
+unless (build_sourceonly) {
+	withecho(@debian_rules, $buildtarget);
+	withecho(@rootcommand, @debian_rules, $binarytarget);
+}
+
+my $signerrors;
+if ($iter == $nloops) {
+	if ($usepause 
+	($signchanges ||

Bug#671074: dpkg-buildpackage: option for building multiple times in a row

2012-05-01 Thread Jonathan Nieder 
Hi Jakub,

Jakub Wilk wrote:

 The attached patch adds option to build package multiple times in a
 row. This is useful for testing correctness of the clean target.

 The patch looks noisy, because I had to indent a few dozen of lines;
 but other than that, it adds just a few lines.

 If case you wonder why can't I just call dpkg-buildpackage in a
 loop: I can't if run it through a wrapper like sbuild or
 $VCS-buildpackage.

Neat --- sounds like a useful test.

I'm not convinced this is worth a dpkg-buildpackage option on its own.
Wouldn't other sequences of targets be worth testing, too?  (E.g.,
debian/rules build  debian/rules clean  fakeroot debian/rules
binary.)

I assume in the $VCS-buildpackage case you are building with the
--export-dir option so the build tree is wiped out between builds.

Possible alternative methods:

 a) teaching sbuild and $VCS-buildpackage an option to use another
command instead of dpkg-buildpackage, or

 b) (as a hack, if (a) is too fussy) overriding dpkg-buildpackage with
a wrapper that appears earlier on the $PATH.

Hope that helps,
Jonathan



-- 
To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org