Re: [Fwd: FAIL Time-HiRes-1.9708 x86_64-linux-thread-multi 2.6.20-1.3001.fc6xen]
> On Thu, 15 Nov 2007 07:27:55 +0100, "Rafael Garcia-Suarez" <[EMAIL > PROTECTED]> said: >> (Why do I care? Because I get every other week a report of Time::HiRes >> failing, that's why.) > Yes, and other core tests are sensitive to load (stress tests for > threads, Benchmark.pm, ...). So that would be useful. But since that > probably needs to be discussed at the TAP level, please followup-to > perl-qa. Why is nobody adjusting the time expectations? When I build perl with threads support I run into test failure far too often. Maybe there is really a bug? This is not only a TAP issue, it must be decided about better values for what the thresholds expected by the tests should be. To me it seems they are wrong. -- andreas
Re: [Fwd: FAIL Time-HiRes-1.9708 x86_64-linux-thread-multi 2.6.20-1.3001.fc6xen]
On Fri, 16 Nov 2007 08:30:49 +0100, [EMAIL PROTECTED] (Andreas J. Koenig) wrote: > > On Thu, 15 Nov 2007 07:27:55 +0100, "Rafael Garcia-Suarez" <[EMAIL > > PROTECTED]> said: > > >> (Why do I care? Because I get every other week a report of Time::HiRes > >> failing, that's why.) > > > Yes, and other core tests are sensitive to load (stress tests for > > threads, Benchmark.pm, ...). So that would be useful. But since that > > probably needs to be discussed at the TAP level, please followup-to > > perl-qa. > > Why is nobody adjusting the time expectations? > > When I build perl with threads support I run into test failure far too > often. Maybe there is really a bug? This is not only a TAP issue, it > must be decided about better values for what the thresholds expected > by the tests should be. To me it seems they are wrong. They are an ongoing issue of debate. If you change the timing to match expectations on a brand new 16 CPU fast Linux machine with lots of fast memory, be sure to break the tests on slow small configured old single CPU Solaris, HP-UX, or AIX machines. It might be more realistic to build in some set_delay () function that sets a basic timing delay in advance, that can be used throughout the rest of the tests, so all tests use the same relative delay. That was just a brain fart, shoot -- H.Merijn Brand Amsterdam Perl Mongers (http://amsterdam.pm.org/) using & porting perl 5.6.2, 5.8.x, 5.10.x on HP-UX 10.20, 11.00, 11.11, & 11.23, SuSE 10.1 & 10.2, AIX 5.2, and Cygwin. http://qa.perl.org http://mirrors.develooper.com/hpux/http://www.test-smoke.org http://www.goldmark.org/jeff/stupid-disclaimers/
Re: [Fwd: FAIL Time-HiRes-1.9708 x86_64-linux-thread-multi 2.6.20-1.3001.fc6xen]
On 15/11/2007, Jarkko Hietaniemi <[EMAIL PROTECTED]> wrote: > Maybe we need a way to mark a test or or a set of tests as 'can fail due > to load, bad randomness, and other fluctuating factors, please try at > least N times (with random sleeps in between) before giving up'? > > This goes for all of t/TEST, t/harness, and the various test modules. > > The other alternative of various module authors having to cook up their > own retry policies would suck majorly. > > (Why do I care? Because I get every other week a report of Time::HiRes > failing, that's why.) Yes, and other core tests are sensitive to load (stress tests for threads, Benchmark.pm, ...). So that would be useful. But since that probably needs to be discussed at the TAP level, please followup-to perl-qa.
