# from David Golden
# on Thursday 18 September 2008 10:23:
>On Thu, Sep 18, 2008 at 12:27 PM, David E. Wheeler
<[EMAIL PROTECTED]> wrote:
>> Can you use the svn copy and not send test results, just to see what
>> happens?
>
>Sure. I can just save them to files with
>Test::Reporter::Transport::File, but as Eric (?) pointed out, that
>doesn't really give us an easy base for comparison. I'll set that up
>either tonight or over the weekend.
First: thank you. To avoid getting a release wedged on some platform
or perl version, we definitely need robots to poke us when we break
stuff.
By 'for comparison', do you mean vs the last official release and/or
0.2808_01 which was in 5.10.0?
My ideal (assuming anyone has the time to approximate it) vision of a
smoke process would be to have many platforms and configurations doing:
A. Run the M::B tests on every commit and send errors to $X.
B. Run through a handful (or all) Build.PL dists on every commit
(where known pass/fail status is available from a previous
release) to detect when we've changed behavior (i.e. lacking
coverage) and send errors to $X. For "handful", see e.g.
the 'onceper' file and perhaps "all" is triggered by something
less frequent than every commit.
http://scratchcomputing.com/tmp/generated_by.module_build.list
http://scratchcomputing.com/tmp/generated_by.module_build.onceper
http://scratchcomputing.com/tmp/generated_by.module_build.therest
$X = the mailing list (or if that is too noisy - provide an easy feed
which many willing and able people are known to be watching.)
Did anyone look at David Cantrell's smoker results?
(http://hetzner.barnyard.co.uk/misc/mb-test-results.tar.gz) The
T::R::T::File setup creates these .db files? If I'm reading that
correctly, the basic "alarm metric" here is a difference between "$x$y"
correlated by $z where:
($x,$y,$z) = split(/ /, $line);
This means we know about the problem and what caused it immediately,
which makes fixing it a small matter of programming (and with momentum)
and zero archeology.
If someone has sufficient time and endpoints to setup something here,
that would be great. (Perhaps a http PUT of the tarball using simply
digest auth would allow smokers to send a batch of results with cron,
which could then be processed and the appropriate alarms sounded by the
receiving server - or something like that. BTW, possibly see Andy
Armstrong's setup for TAP::Harness - though I think that relies on the
svn being on his box and a fair amount of ssh-ery.)
If we have to pass tarballs and scripts around by mail to get this
release out, so be it.
Whatever gets legs works for me as long as we can reach a sustainable
mode where everyone can assume that things are fine unless the robot is
poking them with a stick.
Thanks,
Eric
--
hobgoblin n 1: (folklore) a small grotesque supernatural creature that
makes trouble for human beings
---------------------------------------------------
http://scratchcomputing.com
---------------------------------------------------
