Bug#731940: buildd.debian.org: consider stderr as well as stdout before timeout
Hi,
On Wed, 11 Dec 2013 14:07:14 +0100 Nicolas Boulenguez
wrote:
> Some (admittedly faulty) build tools produce messages on stderr instead of
> stdout.
I don't think it can be called faulty to produce messages on stderr. I'd not
want, for example a progress meter on stdout because that would mean that the
actual program output on stdout that I might want to pipe to another process
gets destroyed.
So I'd argue that status messages should especially sent to stderr and not
stdout. And it would then follow that for sbuild to check whether the build is
still alive it should especially also consider stderr.
> However, sbuild only considers stdout to detect timeout and kill inactive
> builds.
>
> This results in hard to diagnose failures, as the log (rightfully) mixes
> stout and stderr.
This is weird because in the man page of sbuild.conf it says in the section of
STALLED_PKG_TIMEOUT:
>> Time (in minutes) of inactivity after which a build is terminated. Activity
>> is measured by output to the log file.
stdout and stderr are mixed in the log file so the above makes me believe that
stderr would be considered as well as stdout for the timeout.
> For an example, the gnat-gps package has been randomly failing to build from
> scratch for years. Various solutions have been tried for years, causing this
> big package to be reuploaded and rebuild for years. (of course, gnat-gps is
> now corrected)
In botch, which builds very slowly on architectures without support for native
OCaml, I use the following Makefile wrapper around the slow parts to make the
build succeed:
sh -c 'starttime=`date +%s`; \
while sleep 300; do \
now=`date +%s`; \
diff=$$(((now-starttime)/60)); \
echo "running for $$diff minutes and still alive"; \
done' & PID=$$!; \
do_some_stuff_that_takes_ages_on_the_wrong_architectures; \
kill $$PID
I'd love to drop this bit because if the build is *really* stalled then it will
wrongfully never quit.
> Please avoid this waste of human and computer time with something like
>
> line 1500 of lib/Sbuild/BUILD.pm.
> -my $pipe = $self->get('Session')->pipe_command($command);
> +my $pipe = $self->get('Session')->pipe_command(
> +{ COMMAND => [$command],
> + STREAMERR => \*STDOUT});
That patch is not correct anymore and will only result in a failure when trying
to run sbuild with it. The following works:
--- a/lib/Sbuild/Build.pm
+++ b/lib/Sbuild/Build.pm
@@ -1595,7 +1595,8 @@ sub build {
USER => $self->get_conf('BUILD_USER'),
SETSID => 1,
PRIORITY => 0,
- DIR => $bdir
+ DIR => $bdir,
+ STREAMERR => \*STDOUT,
};
my $pipe = $self->get('Session')->pipe_command($command);
Please consider applying it to sbuild.
Thanks!
cheers, josch
signature.asc
Description: signature
Bug#731940: buildd.debian.org: consider stderr as well as stdout before timeout
On Sun, Feb 15, 2015 at 10:30:11AM +0100, Johannes Schauer wrote: > > I'm CCing the sbuild maintainers because it probably needs to be fixed there > > first. > Should instead this bug not be reassigned to sbuild? > This bug even has a patch - fixing it should be trivial. Yeah, fix it in mainline sbuild. Kind regards Philipp Kern signature.asc Description: Digital signature
Bug#731940: buildd.debian.org: consider stderr as well as stdout before timeout
Hi, On Fri, 11 Jul 2014 12:56:57 -0400 Felipe Sateler wrote: > I just spent some time wondering about this due to a build failure in > ardour3. This is a big time waster for everyone involved. during the last three months I uploaded botch with different changes to somehow not cause timeouts on architectures without ocamlopt (= everything will take super long). This problem is especially frustrating because the build failures are not deterministic or might not happen at all. So after I upload a change in the hope it would fix the timeout problem, there is no clear indication whether it did succeed or not. I only now discovered this bug report and it solves the problem caused by my naive assumption that output to stderr would be sufficient to prevent a timeout. As Felipe pointed out this is *highly* frustrating and a big time waster for myself, the buildds and the mirrors that have to repeatedly update a large package that was re-uploaded just to try if some change fixes this problem... > I'm CCing the sbuild maintainers because it probably needs to be fixed there > first. Should instead this bug not be reassigned to sbuild? This bug even has a patch - fixing it should be trivial. cheers, josch signature.asc Description: signature
Bug#731940: buildd.debian.org: consider stderr as well as stdout before timeout
Package: buildd.debian.org Followup-For: Bug #731940 Hi, Nicolas Boulenguez wrote: > Some (admittedly faulty) build tools produce messages on stderr > instead of stdout. However, sbuild only considers stdout to detect > timeout and kill inactive builds. I just spent some time wondering about this due to a build failure in ardour3. This is a big time waster for everyone involved. I'm CCing the sbuild maintainers because it probably needs to be fixed there first. -- System Information: Debian Release: jessie/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'stable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.14-1-amd64 (SMP w/4 CPU cores) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#731940: buildd.debian.org: consider stderr as well as stdout before timeout
Package: buildd.debian.org
Severity: wishlist
Tags: patch
Hello.
Some (admittedly faulty) build tools produce messages on stderr
instead of stdout. However, sbuild only considers stdout to detect
timeout and kill inactive builds.
This results in hard to diagnose failures, as the log (rightfully)
mixes stout and stderr. For an example, the gnat-gps package has been
randomly failing to build from scratch for years. Various solutions
have been tried for years, causing this big package to be reuploaded
and rebuild for years. (of course, gnat-gps is now corrected)
Please avoid this waste of human and computer time with something like
line 1500 of lib/Sbuild/BUILD.pm.
-my $pipe = $self->get('Session')->pipe_command($command);
+my $pipe = $self->get('Session')->pipe_command(
+{ COMMAND => [$command],
+ STREAMERR => \*STDOUT});
Or, if you do not want to hide the problem, please allow the
maintainer to easily understand the problem:
line 1553-1555
$self->log("Build killed with signal " . $timeout_sigs[$i] .
" after " . int($timeout_times[$i]/60) .
-" minutes of inactivity\n");
+" minutes of inactivity on stdout (stderr is
+ignored).\n");
Thanks.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
